July 9, 2007
Split Testing – Trial by Fire
Image by Sebastian Bergmann
Split testing is a cheap and reliable way to test two or more versions of a design against each other and see how they perform under live conditions. When split testing you focus one or a couple of quantitative metrics (such as like revenue, number of completed sales or sign-ups) and use them to judge how each design performs. It is a great method if you want to try out an advertising campaign, a set of rewritten purchase instructions or a new sign-up process. Sure, the method has its flaws (which I will get to later) but it is still a great method for finetuning a web page that every web developer should have in her toolbox.
In this post I will be outlining the basics concepts and list the pros and cons of the two main methods for split testing – A/B Testing and Multivariate Testing.