Multivariate testing, as it pertains to Internet marketing, is a process of testing multiple parts of a live web page or website for the purpose of finding out which combination works best to achieve the goals set out for the site.

Multivariate testing is more complex than standard split-testing, also  known as A/B testing. In A/B testing, only two variations are tested, but multivariate testing includes many different variables.

Marketers generally use multivariate testing to find out which variation of a particular website performs best. This is most often used to maximize the response of a sales page or squeeze page, but some marketers also use this type of testing in other ways, such as for maximizing AdSense revenue through color changes, ad placement, and other factors.

Multivariate testing can be performed manually by changing one thing at a time and recording the results, but this is extremely complex and time-consuming. Most people prefer to use special software to perform this kind of testing.

The software will display differentversions of a page dynamically, and statistics are then recorded about visitor behavior. Testing can also be taken further to dynamically display the version that is performing best at a given time.

Some marketers use this type of system to rotate various versions, displaying the most effective version the majority of the time and displaying the alternate versions only for testing purposes. Since traffic sources, seasons, time of the day, and other factors have an effect on how people react, this can be useful for ensuring the best response at any given time.

Very good testing software can actually track visitors to ensure that they’re shown the same content when they return to the site. It could also be used to deliver different versions to people who didn’t take action on their first visit, with the purpose of delivering a different experience in order to have a
better chance of getting the visitor to take action.

Even PPC companies like Google AdWords are now offering their own style of split testing. AdWords will let you test multiple versions of your ads, and then they’ll display only the most productive of those ads.

A/B testing is generally easier to set up and manage than multivariate testing. You can use A/B testing to test the difference in conversion rates between two headlines, for example, simply by making two different versions of your sales page.

You would run one version until you received a certain amount of data, and then run the second one until you received a similar amount of data. Once you received enough data for the test to be considered statistically accurate, you would compare the results.

No software is needed for most A/ B split-testing, other than a statistics program to monitor the amount of traffic received to each version and a way to track the response rate of the traffic. As long as you can find out how much traffic you’ve received during the time each version was run and how many actions were taken, you will have a successful A/B test.

A/B testing is also very easy to interpret. Since there are only two possible outcomes, the results are usually very clear. Since A/B testing is so straightforward, it can be an easy way for you to test different options if you’re not very familiar with testing or don’t have the resources to devote to purchasing special software.

If you’re a home-based business owner with a simple website and a low budget, multivariate testing may be overkill for you. In this case, setting up a basic A/B test can save you a lot of time and resources.

You might need to test only a small element of a well-performing page, Buy button or the color of your headline. Or perhaps you want to add a new option to a page, but you’re unsure how your traffic will react. You could set up a simple A/B split test in order to find out how the change is received.

On the other hand, multivariate testing is very helpful when you’re creating a completely new website and you want to test various elements to find a version that will receive the bestconversion rate. It can also be helpful if you’ve received a number of conflicting suggestions from your customers that you’re interested in implementing, but you want to test the variations before  ommitting to a specific version.

The most famous type of multivariate testing is known as the Taguchi method. It was first used in the 1950s to test several types of product manufacturing systems, including automobile manufacturing.

These days, the Taguchi Method is being used to test page elements for web pages, and to find out which combination of those variables produces the page with the best results.

More companies are starting to use this method to test web page results. There’s one problem with this method, though. Because there can be so many unique variations, it can take a very large amount of traffic to come up with statistically valid results. If you created a multivariate test that was set
up to compare results with five different variables each with three values – that would make 243 possible combinations.

Since you’d probably need at least 100 conversions for the results to be considered statistically valid, you’d need almost 500,000 visitors if your conversion rate was 5%. Most websites don’t have that much traffic, making this type of testing very difficult to perform.

For this reason, many smaller companies choose to stick with simple A/B testing. A/B testing doesn’t offer the type of flexibility that comes with something like the Taguchi method, but it’s much easier to get statistically valid results with less traffic.

Although A/B testing is simple to set up, it can actually be quite timeconsuming if you have more than two variations to test. Multivariate testing is far better if you need to test several variables, because, although it’s a bit more difficult to set up, it’s far easier to monitor and track when you have software that records your statistics, swaps pages for you, and handles everything for you once it’s been set up.

Some marketers don’t see the need for any type of split-testing. They believe they know their market, and they understand exactly what they’re looking for. While that may be true, it usually isn’t.

Many of the smartest marketers in the world have been shocked to discover that something they believe in firmly actually wasn’t true at all once they performed a split-test! There’s a story about a marketing agency that decided to test three different versions of a particular ad.

The group was asked which ad they felt would perform the best of the three. About half of them voted for one version, and the other half voted for another. No one voted for the third choice. The marketers all cites their reasons for believing the ad they chose would perform well, and the reasons they gave all made sense.

But when the results came in, the entire team was shocked. The third ad was by far the best performer! This was the ad that none of them picked. This was a group of professional marketers with decades of experience, yet they failed to correctly anticipate the reaction of this market. After spending a lot of time researching the market as thoroughly as possible, they still didn’t have a firm grasp on what this particular market wanted.

Skype: An Interesting Case Study

Some of the largest websites in the world use split-testing and multivariate testing in order to improve visitor experience and increase conversions. Even companies like CNet and Skype have used split-testing.

Skype chose to outsource their split-testing to a company called Offermatica. They felt their in-house team or a retained agency couldn’t be objective enough to perform proper testing, so they hired it out to specialists.

Skype had three main objectives:

They wanted to increase the number of people who downloaded the software, increase the number of people who subscribed to their paid service, and measure various calls-to-action and branding methodologies.

Skype used a version of multivariate testing often called A/B/C split-testing. Rather than testing two variables at a time as with A/B splittesting, A/B/C split-testing uses three.Skype decided to test two variations of their homepage against their original version.

They used their original homepage as the control subject for the test. The second version reduced the branding level that was used on the page. The third version used fewer graphics andtext for a cleaner, more streamlined look.

The results were surprising! Although visitors claimed that they like the second version the most, the results actually showed that the response was best with version three, the minimalist version.

Skype also used split-testing to test other elements of their site. They wanted to test the navigation menus to find out the best wording for certain elements in the menus. The results of the testing showed that two minor changes to the wording in the navigation menus increase revenue by a staggering 18.75%!

This type of case study is an excellent indicator of how split-testing and multivariate testing can be used to increase response rates and conversion rates. This is also a very good illustration of the fact that sometimes even the viewers don’t know what they prefer.

They might think they like a certain version better because the colors or layout looks more appealing, but in reality they might respond better to other version. This is why the most successful webmasters are testing, testing, testing – it gets results!

Related posts:

1. Social Bookmarking – Hot or Not? The top Social Bookmarking sites continue on a slight but...
2. How to Write an Effective Adwords Ad Advertising is the name of the game in any business....
3. How To Record Skype Calls and Podcast Interviews Have you ever been in a spot where you were...