A/B Testing: A Coin with Two Sides

Nowadays, businesses have more customer data available to them than ever before. With so many possible metrics to record and analyse, data is an incredible asset for almost all aspects of your company. However, all of this gathered information is only useful if we're using it in the right way. With this in mind, in this post we'll go over one marketing tool that uses all kinds of data to improve a business' overall strategy: AB testing.

How AB testing is defined:

AB testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better. You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins! While AB testing is predominantly used in the digital world, it is actually helpful to businesses in any industry.

From a marketing perspective, AB testing always has two sides:

• AB testing before an event will occur

• AB testing after an event has occurred

To illustrate this better, here are a couple of examples of AB testing showing the two different stages:

 

Example 1

 

Stage 1

Optimize the ebook landing page to achieve a higher conversion rate.

 

Stage 2

Optimize the ebook content so the reader spends at least 4 minutes reading it.

 

The idea

If I have 1000 downloads per day, and nobody engages with my ebook, then the whole campaign is a fail. That’s why it’s important to AB test both stages.

 

Example 2

 

Stage 1

Optimize the CTA on my blog post to gain a conversion rate of at least 5% from blog post to signup completed.

 

Stage 2

Redirect the signups to a schedule link where they can choose a preferred time to talk. Because they are blog readers, they may not understand everything in my app, that's why I’ll call them and explain the product in their language.

 

The idea

If I had 100 signups per day, yet none of them completes the application onboarding, then I’ve lost all 100 signups. That’s why it’s important to redirect blog signups to a schedule link so I’m able to answer their queries directly.

But there is another issue with AB testing...

We talked about optimizing before and after an event has occurred. But AB testing is not just about design, content, structure - it’s also about what kind of audience is testing your product.

To be honest, the type of audience you’re targeting is more important in AB testing than the content and design are. Why? The audience, in many times, does not need an AB testing if they really want something. For example, if I sell marketing services, and bring hundreds of marketing experts to my website daily, they will all engage with my service without the need for AB testing. However, having the right audience can offer other opportunities, such as testing different price points, packages and features, or determining how all of these customers are finding the site.

• So if I have an audience that is in my target and I create an AB test campaign, we can assume that the AB campaign will have some interesting metrics and will define our next changes for an event.

• But if my audience is not my target and I create an AB test campaign, then we can assume that my AB campaign will fail, but even if the campaign succeeds, later it will fail because I’m doing an AB campaign with the wrong audience.

So we have looked at how AB testing should work and its stages and had a short introduction to audience relevance.

But one question remains unanswered: How can I generate a valid audience for my AB campaign?

In the following guide, we’ll try to answer this question by exploring the points below:

• How to get insights into my current website audience and see if my audience corresponds to my target criteria?

There are many tools that provide website visitor insights, for the following example we’ll use LinkedIn. Please be sure to have a LinkedIn Campaign set up if you want to use the audience tracking feature.

1. Sign in to Campaign Manager.

2. Click the account name you'd like to set up website demographics for or set up a new account.

3. Click the Website Demographics tab in the top navigation bar.

4. Follow the instructions on the screen to get started.

5. After you have added the scripts on your website and once you’ve accumulated enough visitors, select the audience(s) you’d like to view or compare from the dropdowns.

 

Now you can understand more about your audience, and discover what kind of companies and/or individuals visit your site. So, for example, if you sell marketing services to SEO consultants, and LinkedIn shows that 25% of your audience are marketers, you can assume that your audience is valid and could be used for future A/B tests.

• Next question: If I’ve found that my audience is marketing, how do I ensure that marketers will purchase or engage my product before I commit to buying a marketing audience for my AB campaign?

The best way to validate LinkedIn data is to create a campaign on LinkedIn and use the audience you found in Website Demographic data.

1. Sign in to Campaign Manager.

2. Locate the account where you want to create the Text Ad.

   1. Note: If you're advertising for the first time, click Add Account and complete the fields.

3. Below Account name, click the account that holds your campaigns.

4. 

   1. Note: If you have multiple accounts in Campaign Manager select the correct account.

5. Click Create campaign and click Select Text Ads.

6. Complete the fields and click Next.

7. Select a Destination and enter your Text Ad content. When completed, click Save.

8. Add any additional creatives and click Next.

9. Select your target audience by Job Title, Industry or Company Size. In our case, we’ll show ads to all LinkedIn Profiles which have the Job Title as "Marketing Manager".

10. Enter your bid type, bid and budget and click Launch Campaign.

So with this Linkedin Campaign, we verified whether or not LinkedIn Insights were accurate, and if we can depend on this audience for future marketing purposes.

• Next question: As we determined that a marketing audience is right for our product, how can we triple this audience?

As we found the right target, we need to find channels to bring a marketing audience to our website. Bringing a professional audience to our website is very expensive these days, for example:

• AVG Linkedin PPC: 12$

• AVG Facebook PPC: 6$

• AVG Quora PPC: 15$

• AVG Google Search PPC for marketing: 7$

These costs are high due to the fact that a marketing audience is a wide professional audience, targeted heavily by other companies. So, we'll leave the PPC campaign aside and instead suggest some cheaper ideas to reach out to the same audience. These include:

• Buying a Marketing Professional List

• Buying a Marketing Companies List

• Renting a third party audience of Marketing from BlueKay or LiveRamp

• Create Content & Banner Partnership with Marketing Blogs

• Rent audience (based on cookies) from Marketing Blogs

When it comes to these other ways of bringing a qualified audience for an AB campaign, the fastest method is likely be to acquire an email list of marketers or a list of marketing companies. Alternatively, another potentially fruitful option is to place your banner on a high traffic marketing blog. This way you can push this audience to your prepared landing page where you'll be able to experiment with different prices, features, packages against a targeted audience.

Creating an AB campaign is enjoyable when you see positive results, but in a market where companies compete for the smallest wins, it's important to set up a balanced campaign with a well-defined target audience and predefined events.

 

If you want to create an AB Campaign using the above ideas, here’s a short step by step guide as our final word on AB testing:

• Define what you want to test (Pricing? Features? Colors? Content? Structure?)

• Define who your audience is (Marketers? SEO? Social Managers?)

• Analyze which audience is engaged with your website using LinkedIn Audience

• Test and validate to determine whether your target audience is the right audience through a LinkedIn paid campaign

• Find a reputable database vendor who is able to provide email or company lists for your target audience

• Prepare your landing page and product for AB Testing (Set up and track all events)

• Send a cold email campaign to the audience you bought

• Analyze the AB testing results and see which variations failed and which have succeeded.

• Apply what you’ve learned from the AB campaign to your future marketing activities.