Continuous A/B Testing to Maximize Conversions

Increase your conversions with these A/B testing case studies

By Alaister Low on June 28th, 2012

5
ab testing case studies

Conversion rate optimization can be difficult as no one can tell you what will increase your conversions. People are able to tell you the best practices and what has worked for them before, but the only real way to find out what increases your conversions is by running your own split tests. With that being said, it’s always great to look at A/B testing case studies and what people have done in the past to give you ideas on what changes you could implement to increase conversions.

Here are some great A/B testing case studies that we have collected from all over the internet. Each one is unique, and a different lesson can be learnt from each case study.

Single step vs multi step checkout – ecommerce

The official Vancouver 2010 Olympic store wanted to convert more sales and went about this by testing different checkout processes. Their original checkout process involved a 4 step checkout. This multi-step checkout involved signing in, adding shipping details, billing and review and finally the receipt.

a/b testing case studies

Original multi-step checkout process

They hypothesised that a single step checkout process would reduce the friction experienced by customers and would help more people convert once they entered the checkout process. The variation was longer in order to include all the required form fields, however it was displayed on a single page.

ecommerce landing page optimization

Single step checkout process

The Vancouver 2010 Olympic store ran an A/B test splitting 50% of the traffic to each variation. The A/B test concluded that the variation with the single-step checkout outperformed the original by a whopping 21.8%. This was determined after only 300 transactions. This is a huge increase in conversions, especially with the change being so far down the conversion funnel. The Vancouver 2010 Olympic store were able to see significant gains in revenue from this single change.

Lessons:

  1. Single changes can make huge impacts in conversions
  2. A/B test elements further down the funnel to make larger revenue gains
  3. People find going through different pages as a lot of friction when converting
  4. People prefer to see everything on a single page as opposed to breaking the process into steps

Source: http://www.getelastic.com/single-vs-two-page-checkout/

Headline, button color and button CTA

Headline

This A/B testing case study is from CareLogger, a health app that allows you to keep track of your diabetes. They tested three different things one after the other and achieved a whopping 72% increase to their conversion rate. The first element on the page they tested was their headline. The original headline read “Keeping tabs on your Diabetes just got a lot easier”. This headline was great. It explained exactly what they offered  - “Keeping tabs on your diabetes”, and provided a benefit as well – “a lot easier”.

headline testing

They decided to run a split test with an alternative headline to see what would happen. The goal they were measuring was the number of free trial sign ups. The variation headline read “Maintain your Optimal Health by keeping tabs on your diabetes”. Notice they still explained exactly what they offered – “keeping tabs on your diabetes”, however they changed the benefit to what seemed to be stronger and more inline with people’s wants – “Maintain your optimimal health”. They also highlighted “Optimal Health” to make it stand out as they believed this was what people wanted to see.

headline optimization

After 1000 conversions they had the second variation converting 31% higher than the original which was substantial for one small change.

Button color

Button color is one of the smallest changes any marketer can make. It’s often one of the most quoted changes when people are trying to show how small changes can have big impacts on conversions. CareLogger decided to give this a go and see what would happen. They had a green Call to Action (CTA) button which they wanted to test with a new red one.

button color

After 600 signups with the exact same CTA and just a different button color the red button saw 34% more conversions than the green.

Button CTA

The third test CareLogger ran was the actual CTA text within the button. The button as you can see above read “Get Started Now”, which was now red in color. They tested this with “Signup for Free” as they hypothesised that people will be more likely to sign up if they are told and reminded that signing up was free. This proved to be correct. After 1000 signups the new CTA text converted 7% higher than the old one.

With these 3 relatively small conversion optimizations CareLogger was able to increase their total conversion rate by 72%.

Lessons and takeaways:

  1. Small changes have big impacts
  2. Emphasize your strongest benefit in your headlines
  3. Find out the real problem your product solves and highlight this to your visitors
  4. Use colors that contrast with each other on your CTA buttons
  5. Costs is always a concern, so if your product/service is free make sure people are aware of this

Source: http://dmix.ca/2010/05/how-we-increased-our-conversion-rate-by-72/

Headline tests from 37 signals

37 Signals wanted to test headlines and subheadlines on their Highrise signup pages. The goal they were measuring was the number of signups for their Highrise product. They came up with 5 combinations of different headlines and subheadlines and ran them all in a single test experiment. Below are the 5 variations and their results after 4000 pageviews had run through the test.

headline split testing

This was the original headline and subheadline combination. This acted as the control for the experiment and actually performed the worst out of all the variations.

conversion rate headline

This headline and subheadline combination tried to use testimonial quotes and a little bit of social proof to show how great the product is. It also emphasized that people could try the product for free. It outperformed the control but only marginally by 7%.

landing page optimization headlines

In this variation the main headline was the same as the control. The subheadline tried to make visitors feel comfortable with the product by explaining that it was “Pay as you go”, “No long term contacts”, “No hidden fees” and “No surprises. It replaced the “30 day free trial” in the control with more comforting features. This variation had a 15% better conversion rate than the original.

headline conversion rate optimization

The next variation used the main headline that told people they could try for free. They mixed this message with the subheadline that made viewers feel comfortable. It shouts out “There are no strings attached”. This variation received 27% more conversion than the original.

split testing headlines

The overall winner in this experiment was this variation with the headline that told people about the free trial. It told people they could try the service for free for 30 days. The subheadline explained how easy it is to get started – “less than 60 seconds”. This headline and subheadline combination explains how easy it is to get started and that you won’t need to invest any money and only very little time to get started. This variation was the highest converting and achieved 30% more conversions than the control did.

Lessons and takeaways:

  1. Test multiple variations in your experiments
  2. Change one element in each variation
  3. Think of different hypothesises and create variations that back these up
  4. Small changes in text can have large impacts on conversions

Source: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

Overall A/B testing lessons

As you can see from the above A/B testing case studies there are many changes, tweaks and redesigns that can be made in the hope of increasing conversions. The biggest takeaway is to think about the visitors that come to your site and create an intelligent, well informed hypothesis as to what will help with conversions and why. Once you have done this you can go about building your variations, whether they be complete page redesigns or simple button color changes. Landing page and conversion rate optmization is a science and it can be a long process. Spend time running your tests properly and you will see great results.

Tags: , , ,


About the Author


Alaister Low is a co-founder of Growth Giant which is a tool that uses the Multi-Armed Bandit Algorithm to continuously test landing pages. It automatically directs traffic to your better performing pages to ensure you get the maximum conversion rate. Google+


Back to Top ↑