A/B testing in a nutshell(Free beginner guide)

What is an A/B test?

When you run an A/B test, you compare one page against one or more variations that contain one major difference in an element of the control page. After a set amount of time, or visits, you compare the results to how the change affected your results.When I learned A/B testing from CXL and found A/B testing is actually a nickname for a variety of testing that involves single element changes across multiple variations, so it might actually be A/B/C/D testing, but it is still called A/B testing and if you see A/B/n or Split Test that is just a more accurate expression of one control with many variations. Every visitor will see one version of the page or another, and you’ll measure conversions from each set of visitors. A/B tests allow you to test one version of copy, images, forms etc. against another. Layout choices such as where you place the elements can also be tested, or functionality choices like when and how you show error messages

How Much Traffic do You Need for a Valid Test? A common question is how much traffic you need when running a test. This will depend on a few factors. If you can run 100–200 visitors to your pages, you can sometimes make gut decisions about pages that are obviously underperforming and initate a new test. But generally you’ll want to wait for about 1,000 total visitors and make sure you run the test for a week to cover daily variances. Use A/B Test Results for Major Site-Wide Decisions Split tests can also be used to temper drastic changes like redesigning a homepage by only serving it to a portion of your visitors. Companies like Twitter and Facebook use this strategy to test major interface changes by only rolling out the new version to a segment of their visitors and measuring how that group reacts. A/B test results are influential in making good decisions in site redesign because they help you see what elements are important for your audience

and customers in a controlled environment. Once you have seen the results of a test you can take the new knowledge to your site wide design. One of the best things about A/B testing is that it can be very fast and dramatic. It is also less complicated to set-up and reach significant results in order to make informed decisions. Advantages of A/B Testing They are fast Advanced analytics can be installed and evaluated for each variation (e.g. click tracking, heatmaps, etc.) Can achieve more dramatic conversion rate lift results Requires less traffic Disadvantages of A/B Testing More dramatic failures Less specific understanding of element effects

Getting started with A/B Testing This is one of those things that is hard for a lot of people. How do you start? The simple answer is to make one page detailing the simple facts about your product or service. These simple facts are the features of your product. These features have consequences. Those consequences are benefits. Secondly, ask yourself why are you better than the other options (the competition) for solving the problem? Is it a lower price, better shipping, better return policy, etc? Online you have two primary ways to communicate information: image and copy. All good pages are going to cover the following 5 things: 1. A headline with a Unique Value Proposition 2. A demonstration of the product or the benefit — preferably in the context of how it would be used 3. A clear value for the viewer 4. Support of value 5. One action Your action and your headline are the only things that need to be copy driven, everything else is wide open. When you create your control page, make sure all 5 of these elements are represented. If you have done this you will have a solid foundation for testing

Building a Control Page Control pages are your first step in creating a generation of tests. Your control is the parent and variations are the children. The winning variation becomes the next generation’s control for further variation. A good place to start your control page is a wireframe. Take a look at KissMetrics’ Landing Page Blueprint (seen on the right). But, don’t let that stop you from using your own vision in how you layout your elements. Just because a certain structure is popular doesn’t mean it’s right for you, also, standing out can be a very effective strategy for good results. Once your basic structure and content is in place you will have a starting point for tests. Good structure will make a world of difference in your testing success, because it will make each change less time consuming

Building Page Variations Each variation should have one hypothesis of change: color of background, different major image, different CTA in the headline, different CTA in the action button, etc. It doesn’t matter how many hypotheses you have, but you will always get the best data by only having one hypothesis per variation. Once you get used to how your target market behaves you may be able to get away with a multihypothesis test, but when you start out stick to a framework that maximizes test cleanliness

into designing your test, if you are communicating to the wrong audience, you will acquire the wrong customers or leads. Keep an eye on what your new customers or leads do, and if they’re not the right ones, run a new test with different messaging to be more inline with your most important business goals. A good technique for tracking your test performance is to keep a record of your hypotheses and your results, so you know where you’re going right and where you went wrong. That will make your next test better, and serve as a record for stakeholders, as well as yourself. Note: You should use descriptive experiments names like Form Length 2012 Catalog or Call To Action: Registration, something that makes the test parameters apparent, not Steve’s Test or PPC Test.