A Digital Insider's Tips for Flawless Split Testing

Share This Content

One of the most common concerns for advertisers is making sure they are getting the highest-possible return on investment. This makes “split testing” one of the most effective ways to maximize your ad dollars. Also known as A/B testing, split testing is an imperative part of any successful digital marketing strategy. Through well-planned and executed experimentation, digital marketers are able to use split testing to inform strategy, improve performance and find the most successful ways to reach your audience. Split testing can be used on PPC ads, collateral, landing pages, web page layouts, content and even email marketing. For the purpose of this article, we’ll be focusing on split testing for digital ads.

What is Split Testing?

At its core, a split test is a form of experiment where a marketer will compare two advertisements, one a control and the other a variation, to determine which is more effective. Once one variation is found to be more effective at reaching marketing goals than the other, the better-performing ad can be continued, while the poorer performer is replaced.

For example, we could run two advertisements with identical content and imagery, but simply change the call to action so one reads “Shop Now” and the other reads “Learn More.”


Once the two ads have run for long enough to collect a reliable data set, you can determine which performed better, and use that knowledge to improve the overall campaign. In this example, we may find that “Shop Now” ends up gaining a higher number of people actually buying shoes. Hence, we would continue running the top performer, and remove the ad with the “Learn More” call to action.

Analyze, Iterate and Refine
With every marketing campaign, it is important to identify the key performance indicators, or KPIs, that you will use to determine performance or return. In a digital ad, these KPIs are often click through rate, cost per click, cost per conversion or, in some cases, cost per thousand impressions. Identifying your KPIs ahead of time is the best way to make sure you can objectively determine performance in a split test.

The real key to split testing is understanding that it’s never finished – there is no end date. After each test, the marketer needs to analyze the data against their KPIs to determine which version performed better, set up a test with another small change or iteration, and continue that process of refining the ads in an ongoing manner. Over time, a campaign with a strong split testing strategy will grow many small improvements until it’s running smoothly and delivering maximum value.

Common Pitfalls of Split Testing
It’s worth noting that poorly performed split testing can actually negatively impact your marketing campaign. Without careful planning, it can lead to misleading results. Most of the common mistakes of A/B testing comes down to failing to properly control for additional factors, and not having a firm grasp on the data set. When performing a split test, it is important that only one variable is changed, while everything else remains constant.

In the example above, the only factor being tested was the call to action. If the ads had also shown a different image, it would only muddle the data, leading to unclear results. What led to higher performance? Was it the use of a different image? The call to action? The combination of the two?

Another common mistake is running the ads at different times or showing them to different audiences. For example, if we run the “Learn More” ad in October and the “Shop Now” ad in November, we can’t be sure the call to action was behind the increased effectiveness of your ad. Maybe users in November were more likely to buy going into the holiday season. Perhaps interest in shoes was higher during Black Friday and Cyber Monday, and our ad variation had no real effect. If control factors aren’t accounted for, the tests will be inconclusive or lead us down the wrong path. Other common factors marketers can fail to control are demographic targets, time of day/month/year and seasonal trends.

Plan, execute, analyze, iterate, refine, repeat. Follow through on this process with an experienced digital marketer and you’ll maximize the success of your digital campaigns. To learn more about how Russell Herder approaches digital marketing projects, see our digital marketing capabilities.