A/B Testing as a Marketing Campaign Tool

A/B testing (bucket tests or split-run testing), refining, and applying is not just an essential part of a business, but of life itself, rooted in the foundations of human beings as an unconscious necessity for survival. You are absolutely not into marketing if you have to depend on nothing but

a hope and prayer as a basis for moving forward with your multi-million-dollar campaign strategy.

Testing has been carried out for a very long time and cuts across different fields in order to determine probable chances and outcomes of events and experiments. In some instances, experimental animals and even human beings are used especially in natural sciences.

A/B testing, or split testing as is applied to marketing, is a special randomized experiment in which you want to choose the best variant of two hypotheses. Marketers basically seek for a way to compare two versions of a single variable, typically by testing the audience response to variant A against variant B, and determining which of the two variants is more effective.

According to Yelp co-founder Jeremy Stoppelman on Innovating and Staying Relevant, “You want experimentation. Every once in a while you stumble upon something that blows your mind.”

Human beings unconsciously and within a few seconds, use informal experiments all the time as a way to test-drive decisions that may have long-reaching consequences. In order to decide whether we will take a certain action, we first construct a hypothesis, run an experiment, analyze the results, and then make a decision based on the available data.

At the end of the day and before eventually embarking on that your intensive marketing campaign, you would feel more confident and secured in that massive business initiative if you’d already found enormous success within a test sample. Once you fail to do this, you only succeed in making a wasteful process more efficient and you end up getting better at doing something the customer does not even want and will never cherish.

Chris Kostecki, in-house Search Analyst at  Keurig Inc, reveals that the application of the outcome of A/B testing resulted in a stronger conversion rate (+18% with 95% stat significance reached) and a higher AOV. Usually, the favorable outcome of the two variants when applied to your marketing campaign serves as the cardinal point, compass, or the focal point for your brand.

So, how does A/B testing enhance marketing campaign?

1. Creativity

We have gone past the stage where marketing is all about the mastering the use of traditional channels, brands are constantly innovating and utilizing creative solutions that will maximize returns.  Creativity is critical to differentiate one’s message, grab people’s attention, and effectively reach the various audiences that are being targeted.

Major marketing decisions you take will have to change as companies are forced to shift focus and reach people where they are. It’s absolutely necessary that you understand what the customer wants and make concerted efforts to align with it if you want to be relevant in the market.

Where you have two hypotheses, A/B testing of the variants comes to the fore. Derek Hess described it best: “We all have different paths to the same destination.” Curiosity leads to continuous improvement.

You may find it more comfortable to dwell on what is in vogue but that will be the wrong way to rationalize. Don’t allow yourself to be caught spending too much time being reactive. By shifting your mindset to what you need to do to grow and questioning today’s tactics and routine, divergent thinking begins.

2. Return on investment (ROI)

The whole idea behind Lean Marketing is to cut wastes and make better profits. The Lean approach begins with a detailed understanding of what value the customer assigns to product and services.

At the onset, it will seem as if A/B testing or carrying out any experimentation adds unnecessarily to your budget since human and financial resources would be sunk in. When you fail to make this the core of your brand, you only succeed in making a wasteful process more efficient and you

end up getting better at doing something the customer does not even want and will never cherish.

It’s a truth that measuring ROI of optimization is hard but a glimpse into what it can do was given by Amelia Showalter, former Director of Digital Analytics for Obama for America, that explains how hard it is to track and measure everything, at least in the long term:

“When we’re working on the campaign, we’re actually working so hard to run all those tests that we didn’t always keep perfect track of exactly what results were long term. It’s hard to calculate this stuff out when we want to put all our resources into running more tests. So, we don’t actually ever have a perfect estimate of actually how much extra revenue was due to our testing, but I think that $200 million is a fairly reasonable estimate.”

3. Customers’ input

With A/B testing, your customers’ views will be your mainstay. It may not sound like a discovery that every company prides itself in giving customers what they ask for, but where you may be getting it all wrong is that customers often describe the solutions they want in fewer focus groups and surveys and sit back and wait while R&D rolls up its collective sleeves and gets to work on materializing their ideas.

Your A/B testing should focus on asking your customers for desired outcomes instead of solutions. The bottom line is to turn your customers’ input into innovation.

A/B testing can outrightly be inefficient if not anchored in good experimental design. Brands should rely on their Consumer Insights departments to apply design of experiments discipline to the test design. This can reduce the number of cells needed to isolate the incremental effect of each variable of interest. Fractional factorial designs can give marketers much more bang per buck in any given test while focusing attention on the variables of greatest interest.

Photo Credit: stefanomaggi Flickr via Compfight cc