Once the two ads have run for long enough to collect a reliable data set, you can determine which performed better, and use that knowledge to improve the overall campaign. In this example, we may find that “Shop Now” ends up gaining a higher number of people actually buying shoes. Hence, we would continue running the top performer, and remove the ad with the “Learn More” call to action.Analyze, Iterate and Refine
With every marketing campaign, it is important to identify the key performance indicators, or KPIs, that you will use to determine performance or return. In a digital ad, these KPIs are often click through rate, cost per click, cost per conversion or, in some cases, cost per thousand impressions. Identifying your KPIs ahead of time is the best way to make sure you can objectively determine performance in a split test.
The real key to split testing is understanding that it’s never finished – there is no end date. After each test, the marketer needs to analyze the data against their KPIs to determine which version performed better, set up a test with another small change or iteration, and continue that process of refining the ads in an ongoing manner. Over time, a campaign with a strong split testing strategy will grow many small improvements until it’s running smoothly and delivering maximum value.Common Pitfalls of Split Testing
It’s worth noting that poorly performed split testing can actually negatively impact your marketing campaign. Without careful planning, it can lead to misleading results. Most of the common mistakes of A/B testing comes down to failing to properly control for additional factors, and not having a firm grasp on the data set. When performing a split test, it is important that only one variable is changed, while everything else remains constant.
In the example above, the only factor being tested was the call to action. If the ads had also shown a different image, it would only muddle the data, leading to unclear results. What led to higher performance? Was it the use of a different image? The call to action? The combination of the two?
Another common mistake is running the ads at different times or showing them to different audiences. For example, if we run the “Learn More” ad in October and the “Shop Now” ad in November, we can’t be sure the call to action was behind the increased effectiveness of your ad. Maybe users in November were more likely to buy going into the holiday season. Perhaps interest in shoes was higher during Black Friday and Cyber Monday, and our ad variation had no real effect. If control factors aren’t accounted for, the tests will be inconclusive or lead us down the wrong path. Other common factors marketers can fail to control are demographic targets, time of day/month/year and seasonal trends.
Plan, execute, analyze, iterate, refine, repeat. Follow through on this process with an experienced digital marketer and you’ll maximize the success of your digital campaigns. To learn more about how Russell Herder approaches digital marketing projects, see our digital marketing capabilities.