How To Do A/B Testing And Go From Beginner To Pro

If you want to be a good marketer, you have to use A/B Testing. You’ll find that A/B testing, also known as split testing, is the foundation for digital advertising. You might find it helpful to consider it a science experiment. There is a hypothesis that you want to see proven, you have your variables, and you have your data. I have seen businesses discover new taglines, value propositions, and target audiences with split testing. I use it every day to find the best performing copy and creative in advertising campaigns on social platforms and on search. I’ll talk about what split testing is, how to get it right, and the common pitfalls of split testing. Then, I’ll talk about a new type of testing that takes A/B testing to a completely new level. Let’s dive in!

What Is A/B Testing?

Split testing is a method in which you test two different variables in your advertising to determine which variable works best.  

Marketers use split testing in just about everything – search ads, social ads, email, and landing pages. You name it, it’s split tested. You’ll find that split testing is effective because there is a rapid learning process involved. Day after day, you get a good hard look at what resonates with your target audience.

Just as importantly, you are able to discard messaging that is ineffective, or worse, presents a negative impression of your brand. In his book, Ogilvy on Advertising, David Ogilvy recounts an instance in which people who hadn’t seen a particular brand’s advertising were more likely to purchase that brand’s products versus people who had seen that brand’s advertising!

How To Get A/B Testing Right
There is a lot that goes into testing, so it’s easy to get confused. Some marketers will utilize split testing, but they won’t test the right items.

The first and most important item of any split test is that you present your ad variables to the exact same audience. You can’t expect to be receiving the right response if you are randomly sending out emails to different groups of people. I’ve included a short list of what to attempt, and what not to, in an ideal split test

Variables To Split Test:
Value Propositions. I believe that this is the first item that every advertisement should look into because it helps to discover the true value of the product. When you test your value propositions, you’ll find the promises that convert into sales. Is it because your vacuum helps clean up more efficiently than other vacuums, or is it because it makes no noise? Only your audience can tell you!
Images. Different people respond to different images, it’s just how it is. The best images are ones that induce curiosity into the viewer. Keep in mind that men respond most often to photos of men and women to photos of women (and babies).
Call To Action. The call to action is what you use to turn a prospect, into a customer. The stronger your call to action is, the more likely you’ll be converting in higher numbers.
– Audiences. Okay, this is the one exception. Some people are more likely to convert than others. If you take the same message and submit it to two different audiences, you are likely to get different results – and a much better idea of who your target customer is. I recommend this variable once you have a good idea of your value proposition.

 

Example: Value Proposition/Benefit Testing on Adwords

We’ll take a look at Zoho, a premium CRM software solution on Adwords.

In this first ad, Zoho’s Headline touts that its CRM will empower your sales team.

In the second, the Headline tells you that Zoho is customizable and user-friendly.

Which one is better? Only click through and conversion data will tell you. The second thing you’ll notice is that the rest of the ad is a lot different, too. This is a good example of a complex but (in our findings) superior type of A/B testing known as multivariate testing, which I’ll get to in a bit.

 

Variables to Ignore (or focus on later):
Length of Copy. One thing that a lot of advertising studies have shown is that the length of copy is not a concern. As long as what you are saying is important and educational, your audience will read as much of it as they can. The other reason why I mention this is because most digital advertising platforms have character limits, so you are likely to be limited anyway.
Word Usage/Play. A lot of marketers think that they’re tricky if they have wording variations in their split tests. Unless you have a significant amount of data and you are trying to squeeze out every last dollar, don’t bother with this.
Colors. I’ll also file this one under “unless you’re trying to squeeze out every last dollar”. Will a blond haired person really get you more sales than a brunette? I’ll admit that it’s possible, but when you’re trying to run an ab test, wouldn’t you rather focus on high impact items?

Now, you don’t have to test out all of the items I mentioned in your split test. However, the more items you mix, the more likely you’ll find the ad combination that brings in the most sales.

The Problem With A/B Testing

Let’s take Ad A, and Ad B, and pit them against each other. Ad A happens to have a 6% conversion rate, and Ad B happens to have a 4% conversion rate. Ad A must be better, it has a higher conversion rate. Sure, in this particular instance. But what if you ran Ad A versus Ad A? Surely, the clickthrough rate for them has to be the same! Well, as you might imagine, it’s not the same. This very concept, to me, puts the idea of just A/B testing on thin ice.

The second problem is that with digital advertising, it can be difficult to evenly divide people across your two tests. For example, people under the age of 30 might have a different reaction to you ad than people over 30, but you won’t know any better. Luckily, some platforms such as Facebook do their best to take this into account, but it’s never perfect. This is why I recommend audience research as a variable as you further develop you’re winning advertisements.

The Best Methods For Split Testing

In my time doing A/B experiments, I have found that you cannot just rely on an A versus a B. To this extent, I now use an average test that will look like this, or what I call “The Average”.

The Average: AA//BB
The beautiful thing about this type of test is that you remove certain problems from normal split testing. For one, you get to ignore the possibility that people might just have been in a better mood when they saw the first ad. You increase your certainty. How do you do this? It’s simple. Take your A test and your B test, and simply add copies of each to your campaign. Then, average out the relevant factors such as click through and conversion to find what is really performing best.

The Boosted: AA/BB/CC
Want to know what your audience responds to even more quickly? Add an additional variable! I find that this type of test works best when you have a variety of value propositions or images that you want to try out.

Multivariate Testing
A new type of A/B test that has become more popular recently is multivariate testing. With multivariate, you combine different headlines, body copy, and images, and combine them in different ways to see what performs best with an audience. If you want to imagine the possibilities, consider 3 Headline variables, 3 Body Copy variables, and 3 Images. You would end up with 27 different ads! As you can imagine, this is quite complex, but you can find strong results and the best combinations more quickly and accurately than other methods.

Traditional A/B testing will likely go out of style in favor of multivariate. As I mentioned before, there are a lot of downsides to traditional split tests: for one, it consumes both time and financial resources when you can only test one variable. Secondly, A/B tests aren’t always authoritative, although my suggestions for averages are the way to go. Multivariate helps you find what works in every way and avoids the traditional pitfalls of A/B when you get to see reactions to every possible combination.

Our team at SPECTRE uses artificial intelligence software to implement these types of experiments. We simply feed the software the different variables, and then develop new campaigns on social and Adwords with the results. It’s the fastest way to a strong ROI, and it tends to be less expensive and it takes less time.

Conclusion
Want to up your game? Start using A/B split testing. Want to ascend to another level entirely? Start using multivariate testing and outperform your competition.

 

 

 

 

 

Cover Photo by Steve Johnson on Unsplash

Share This

Copy Link to Clipboard

Copy