Continuous A/B Testing to Maximize Conversions

Lean Website Optimization: We don’t know anything!

By Alaister Low on July 16th, 2012

lean website optimization diagram

We’re always trying to optimize our websites to improve them and get more customers, but most of the time we’re completely wrong about what we think our customers want and what will actually improve our conversions. “Lean” methodology has been around for a while, it’s a highly efficient way to improve business by creating more value for your customers using fewer resources. Just recently, lean thinking spread through the startup world like wild fire, spurred on by Eric Ries’s highly successful book “The Lean Startup”. We have combined lean thinking with website optimization to create a hybrid way of thinking called “Lean Website Optimization”, a highly efficient way to get more out of your website.

Before we explore this new concept lets firstly take a brief look at The Lean Startup and the concepts it encompasses. The concept of The Lean Startup is to employ continuous innovation to achieve a successful business. It’s based on flexibility, efficiency, adaptability and reacting quickly to what customers want. It involves creating an initial Minimal Viable Product (MVP) to prove a hypothesis to see if people want to buy your product. It revolves around learning customer behavior, their needs and wants, and iterating quickly. This is the new way to build a company.

We’re big believers in The Lean Startup methodology and try to use the theories and concepts taught as much as we can in our own startup. For this reason we have adapted and applied lean principles to website conversion rate optimization to create a new and improved way to optimize your website called “Lean Website Optimization”. Let’s delve into this new way of thinking and discover how it can help you improve your website.

Lean Website Optimization

[leen web-sahyt op-tuh-muh-zey-shuhn] 

A process of continuously testing different hypotheses for conversion rate optimization. This relies on user driven scientific data that can be fed into a feedback loop to create meaningful direction for website owners. The process employs multi-armed bandit theory to achieve maximum conversions as learnings taking place.

Lean Website Optimization is basically a highly efficient way to optimize your website to achieve your goals. It relies on user driven scientific data that can be fed into a feedback loop to help website owners change and improve their websiteIt involves running continuous A/B tests to get a better understanding of what increases conversions on your website and why. Most of the time website owners are completely wrong about what they think their visitors want to see and what will increase conversions. The process of lean website optimization and running continual, efficient A/B tests helps with increasing conversions and understanding your audience at a much deeper level.

Here are some powerful quotes from leading businesses which show how little businesses actually know about their customers, highlighting the importance of testing:

  • In the paper Online Experimentation at Microsoft, they shared the statistic that only one third of ideas tested at Microsoft improved the metric(s) they were designed to improve. 
  • Jim Manzi,  in his book Uncontrolled: The Surprising Payoff of Trial-and- Error for Business, Politics, and Society , writes “Google ran approximately 12,000 randomized experiments in 2009, with [only] about 10 percent of these leading to business changes.
  • Avinash Kaushik, Googler and author of Web Analytics: An Hour a Day, wrote in his Experimentation and Testing primer that “80% of the time you/we are wrong about what a customer wants.”
  • In Do It Wrong Quickly, Mike Moran wrote ”Netflix considers 90% of what they try to be wrong.”

It’s difficult to approach a conversion test with the attitude that we are wrong 80% of the time. However to achieve real progress we need to run tests without favoring variation ideas and really analyze visitors’ behaviours to understand what increases conversions and why. With statistics like these, it’s vital to be continuously testing and that the results be accurate and trustworthy. Incorrect results may cause bad ideas to be deployed or good ideas to be incorrectly ruled out. It would be a complete waste of time and can cause bad businesses decisions to be made.

We have created a simple methodology for lean website optimization that allows you to understand your visitors better and learn how to increase conversions on your website.

lean website optimization diagram

Continuous testing to ensure maximum conversion rate

1. Formulate a Hypothesis

This initial step involves coming up with a hypothesis for increasing conversions. This revolves around perceived customer behaviour, needs and wants. An example of a hypothesis could be, “trust symbols will increase conversions because a pain point for visitors which prevents them from converting is that they don’t trust my site”.

2. Create Variation Ideas

Take the hypothesis above and create a variation of your web page that includes or emphasizes it. The goal is to prove the hypothesis above is correct. Perhaps you want to add a section on your landing page that includes logos of businesses who use your service. Or you might like to add real life case studies and testimonials to your site. This will provide an element of trust as your website visitors can see that real businesses and people are using your product.

3. Continuously A/B Test

Once you have come up with a variation idea, test it out using multi-armed bandit theory. This new way of A/B testing actually allows you to add new variation ideas to your tests at any time and traffic will be dynamically adjusted accordingly. More traffic is sent to better performing pages while poorly performing pages receive less traffic and are eventually phased out of the test. All tests run continuously, encouraging you to keep testing new ideas to optimize your site.

4. Measure Results and Learn

Monitor the conversions and the amount of traffic that is sent to each variation. Watch for variations that are getting phased out of the test due to poor performance and for pages that are getting the majority of the traffic due to high conversions. Compare the two variations and try to come up with conclusions as to why this is happening. For example, a page with a lot of trust elements like testimonials and customer logos may be performing poorly, while a variation with a “how it works” diagram is getting more traffic and converting higher.  This could mean that your visitors are lost and need more of an explanation about how your product works. This provides valuable information that can be fed back into step 1 to formulate another hypothesis.

Advantages of Lean Website Optimization

Lean website optimization as a methodology to increase conversions has a lot of advantages over running traditional A/B tests and conversion optimization strategies.

Maximize conversions at all times

By using multi-armed bandit theory, the risk of losing conversions and revenue during the experiment is minimized. With lean website optimization we are able to reduce the cost or  ”regret” of our test (regret being the difference between optimal conversions from the highest converting page and actual conversions during the experiment). This means that we can maximize our conversion rate and thus achieve a higher average conversion rate than traditional A/B testing.

Develop a deep understanding of customer behaviour

By keeping website optimization lean you are able to develop a deep understanding of your market. You will have scientifically proven data on their needs, wants, behaviour and purchasing habits. This is very valuable for developing new features and products as well as improving customer satisfaction.

Improve customer satisfaction

The lean website optimization methodology allows changes to take place that are the result of user driven data and scientifically proven hypotheses. As a whole, over time this will ensure that your website is built around your customers. Your site will address customers’ exact concerns and issues as well as provide them with the solution they are looking for. Due to the continuous nature of this methodology and the powerful feedback loop, it means your site will continually improve as your customers and market change and adapt. Being able to flexibly change and adapt is a huge competitive advantage.

bill gates ab testing

As you can see this process of continuously A/B testing is a cycle and is possible due to multi-armed bandit theory which we employ in our Growth Giant tool. You are able to add new variations (each with a different hypothesis) to the experiment at any time and our tool will work out the conversion rate and send more traffic to higher converting pages. This maximizes your conversion rate at all times as your traffic isn’t blindly sent to poorly performing variations like in traditional A/B testing. This allows you to spend more time analyzing the behaviour, needs and wants of your visitors so that you can come up with effective web page variation ideas to test and improve your conversions further.

As website business owners, we all know that we should be constantly looking for ways to improve our website, thus we need a continuous process in place to empower us to do so. Lean website optimization, if used correctly can be very effective and provide your business with a huge competitive advantage. You will be able to iterate your landing pages faster for more conversions. This could mean more sign ups, registrations, sales and revenue!


Tags: , , ,

About the Author

Alaister Low is a co-founder of Growth Giant which is a tool that uses the Multi-Armed Bandit Algorithm to continuously test landing pages. It automatically directs traffic to your better performing pages to ensure you get the maximum conversion rate. Google+

Back to Top ↑