Saturday, 5 May 2012

A or B, but not both

The idea behind an A/B test in email marketing is that you send different subject lines, creatives and offers to different groups. And by comparing the open, click-through and conversion rates, you can see which one performs best at achieving your desired outcome.

But if you send both versions to the same person then you won't learn that much. Especially when the test is an offer of 10% or 20% off. You need to send them to different groups of people.

Dumb Brummies





Both emails were received this morning, an hour and a half apart, and were sent to the same email address. Although I should point out that the email with the 20% subject line only referred to a 10% discount in the body copy. So rather than an A/B test snafu, it could simply have been that the first email went out with the wrong subject line so they decided to send it again with the correct subject. Which is even more dumb.