Which One is Right for You?
When a review of your site metrics shows results not as great as you would like, it’s time to consider your options. Many factors can go into the performance of a page and you may need to evaluate different approaches. Testing at this phase can be a great way of determining which direction is right. But, what type of testing method should you use? That depends upon a few things.
Both A/B testing and multivariate testing (MVT) can offer valuable insights when they are appropriately utilized. Let’s take a look at each one to understand further.
What is A/B Testing?
With an A/B test, you can effectively compare two full versions of a page. This does not mean you want to see how a page looks or performs when you change the dominate color in the navigation bar. A full-scale page change includes a completely different layout where multiple elements on the page are treated and displayed differently. One option may even include some elements not included at all on the other version.
What is Multivariate Testing?
As the name implies, this form of testing involves multiple variants, or elements to be changed. It could also be called “micro multivariate testing” because in these situations, the changes to be compared are smaller and impact very specific parts of a page. Headlines, CTAs and images are examples of elements commonly testing in a MVT.
In addition to testing specific elements, a MVT lets you test the combination of changes. If you have 3 headlines and 3 calls to action, you could mix and match to create a total of 9 different test options. This allows you to see not only which headline performed best but which headline-CTA combination performed best.
Multivariate tests give you great power but not in all cases. In order to get a true read on the results, more conversations are needed to justify a MVT test. Ideally you would want at least 100 conversions for every possible variation in order to consider the results accurate.
A One-Two Punch
You could make use of both forms of testing in a sequential manner. A possible scenario would be to utilize A/B testing up front to nail down your primary page layout and elements. From there, you could perform a multivariate test to fine-tune specific items or elements on the page.
A Hybrid Approach
Let’s say you have two “smaller” elements on your page that you want to compare but you don’t have sufficient conversions to warrant a comprehensive MVT test. You could then essentially perform an A/B test but just not with major changes, only with the smaller refinements.
What’s the Point?
No matter what type of testing you want to do, there is something that must come first—your purpose. You should only embark on a test with a clear goal in mind. This should essentially include the identification of a problem that you want to solve and often relates to moving people through your site or effectively stimulating the desired actions. This goal will then serve as the foundation for your measurement as to which option works and why.
