Apples and oranges: the ABCs of A/B testing in under 10 minutes

Posted by Toby Knott
Picture of Toby Knott
on 30 July 2015
Advice How to do marketing Writing

A/B testing, or split testing, is as simple as it sounds. You test two variants (A and B) of an element of a particular piece of content – the leading image of a blog post, say – against each other and record which one performs better using metrics that you care about, such as social shares, conversion rate, etc.

Here's one we made earlier.

The beauty of A/B testing is that you can use it to fine tune all of your content – web pages, landing pages, emails, tweets, blog posts, you name it.

That means, with the right testing methods, you can boost your conversion rates by up to 300 percent. And it's cheap, to boot.

New call-to-action

Easy as A/B C

Let’s say you want to run an A/B test on an email.

  • First you'll need some software, like Google Analytics, HubSpot, or Optimizely, to actually measure the results.
  • Secondly, you'll want to choose the metric, or metrics, you want to measure – opens, conversions, click-throughs, etc.
  • Then you'll want to choose what element of the content you want to test. With email, this would probably be the subject line, layout, promotion, call-to-action, image, or the time at which you send it.

But only test one element at a time. Test more than one and you'll have no idea which element affected the test.

Unlike a blog post headline or landing page call-to-action, emails are time sensitive, so you don't want to be testing for weeks or months to get a statistically significant result. You need to figure out the smallest portion of your total email list and split the two variants between this group to get statistically significant results. You can then send the most successful variant to the rest of the list.

Alternatively, if you're not sending the email to a pre-determined list, but it's part of a workflow – ie it's dependent on the lead fulfilling a certain condition, like filling in a landing page form – then just ensure that half the leads get one email, and half get the other and wait until it's clear which variant is more successful.

But don't stop there. A/B testing isn’t a tool to craft the 'perfect' piece of content, but rather to help you intelligently tweak your marketing output. You should be continually A/B testing and using data not just your emails, but all of your content to keep it relevant and effective.

New call-to-action

Hat tip to nubobo for the photo.

See also: marketing data

Related service: Marketing Strategy