Multivariate Element Test Variations

Part 4 of my MVT testing series follows up element selection with element variation. Element variation is the most creative aspect to MVT. It requires a mix of marketing acumen, an understanding of creative and some jigsaw puzzle skills.

Marketing:
The marketing aspect to element variation presents itself in the themes that you create to test against one another. For example the headline can be constructed to be emotional, aspirational, direct, inquisitive, helpful and any number of other themes. These messages can also be presented with a certain voice like peers, experts, underdogs or others.

Creative:
Creative differentiation is the primary key to testing success. The best test variations will look nothing like one another. Something I do regularly with test designs is stand 5-10 feet back from my monitor and look at the variations to ensure they are distinct from one another. I’m not talking about background color or font style, I’m talking about radical creative differentiation.

Puzzle Skills:
Since we are creating an array these radically different pieces will have to come together in their different combinations and variations and need to present themselves as a unified experience. This is one reason it’s helpful to understand what your array is going to be prior to element creation. Puzzles are fun, just make sure you are not missing any pieces.

Here are some tactics for element variation that I recommend:

Reduction or Addition – Test things being present or not present. There is no better gauge what factor of influence they have on performance. More often than not you will find that addition by subtraction is a way to improve your results.

Content Mix – How much information is needed? How should it be presented? What should it be associated with? Should it link out, have icons, bullets, images? Should it be customizable? Does any of it matter? These are all questions a good MVT can answer.

Choices – The paradox of choice is a rule that I’ve seen proven countless times in testing. Sometimes it takes a test to prove the value. Other times there has to be more than one choice presented. MVT can help determine how many choices, the order of selection, where or if defaults should be present and any number of other questions around selection.

Static/Dynamic – At RAMP Digital we believe simple is the new sexy. That ajax scroll bar is cool but is it more effective than a static image? How about your navigation, buttons, images. What looks good and what performs well are often two different things. Rich media can be incredibly useful and improve digital experiences and performance immensely but the only way to know is to test.

Themes – As mentioned, there are multiple messaging themes that can be tested. This provides really valuable feedback that can be used not only on the web but in other media to find the “voice” that best connects with your target audience or within certain segments. Segmentation is very important when analyzing results based on theme. The classic example is the dating site where the image of a couple performed better with women but the image of a woman alone performed better for men.

In Parts 1-4 of this series I hope I’ve fully covered MVT test design. Please comment or email me if you have any questions or there is something you think I missed.

Now that we’ve designed our tests we’ll get test set-up next.