16 Mar 2017 • BLOG - News
Create The Perfect Email Campaign With Multivariate Testing
16 Mar 2017
It’s happened to all of us. You go into that fancy ice cream shop just around the corner, have a look at the different flavors and think “If only I could taste them all…”. And then, as you have finally narrowed your choice down to those two (or three, or four…) you really want, the guy in the shop asks: “Want to try any?” Oh, the joy! Being able to make an informed decision for something as crucial as your ice cream choice…
But what looks quite straight-forward with one scoop turns into a new challenge when we add another one. Yeah, you might love Blue Moon and Coffee, but do you really think those two go well together? No, the answer is no. Don’t even try, it’s weird. So if you thought getting two scoops of ice cream was easy, I’m here to tell you otherwise. It’s not, mastering the perfect combination takes some time… and an awful lot of testing.
Find the right combination with multivariate testing
OK, but what does this have to do with my Email Marketing?
So far you might be thinking we’ve turned into an ice cream blog (wouldn’t that be nice?), but don’t be fooled. Ice cream can be a metaphor for so many things in life… including email marketing.
A few weeks ago, we talked to you about the secret weapons of email and how testing could help you make the most out of them. We’ve already told you all about A/B (or A/X) testing: creating two (or more) versions of an email, changing just one variable (subject line, from name, CTAs…). That is, choosing the best flavor for your email by trying a few of your favorites (and some you might not think could work).
Now, we’re stepping up our game. We’re going for two scoops… and even a topping. We’ll learn how to find the perfect combination to make our email campaigns even tastier.
So what’s multivariate testing, then?
Glad you asked. Multivariate testing basically consists in testing multiple variations of the same email. Duh.
Briefly explained, when you’re A/B testing an email, you duplicate your template and change just one element, be it the subject line, the banner, the text in the CTA… In multivariate testing, you change two (or more) elements, to analyze how the different combinations of these perform.
Ehmmm… example, please?
Sure. Let’s say you have an awesome campaign to boost your sales and you want more people to click-through to your landing page, so you decide to test:
- The text in the CTA: “Buy now” or “Discover our amazing prices!”?
- The position of the CTA: Before the fold, or at the bottom of your email?
To perform a multivariate test, you’ll create and send four variations (2 texts x 2 positions) of the same email:
- CTA “Buy now”, before the fold.
- CTA “Buy now”, at the bottom.
- CTA “Discover our amazing prices!”, before the fold.
- CTA “Discover our amazing prices!”, at the bottom.
Although this seems quite straight-forward, the test gets a little bit more complicated when you add a third element, but this also means the results are even more powerful.
For example, let’s take this awesome campaign of yours and your big CTA dilemma. Yeah, you could test position and text but… what about color? Not adding color to the combination is like not adding toppings to your ice cream when someone offers – totally unacceptable.
So now we’ve got these three things to test:
- The text in the CTA: “Buy now” or “Discover our amazing prices!”?
- The position of the CTA: Before the fold or at the bottom of your email?
- The color of the CTA: Red or black?
So we sit down, open Passport (or whatever, less cool, email editor you’re using…) and create eight variations. Eight, you might ask? Well, yeah: 2 texts x 2 positions x 2 colors. No, that’s not six. Yep, it’s eight. Don’t worry, we won’t tell anyone about your rusty math skills.
Our eight variations will be:
- CTA “Buy now”, before the fold, red.
- CTA “Buy now”, before the fold, black.
- CTA “Buy now”, at the bottom, red.
- CTA “Buy now”, at the bottom, black.
- CTA “Discover our amazing prices!”, before the fold, red.
- CTA “Discover our amazing prices!”, before the fold, black.
- CTA “Discover our amazing prices!”, at the bottom, red.
- CTA “Discover our amazing prices!”, at the bottom, black.
Get them ready, press Send and wait. When it comes to analyzing the results, you’ll be able to see which of your CTA combinations encouraged more contacts to click. As easy as that.
OK, so can I like… test all the things?
Now, we know you’re about to get carried away and start planning a bunch of crazy options for your next email campaign. Hold on, there! First things first, we need to set some ground rules.
- Consider why you’re testing: And no, ticking the box is not a valid answer. Tests are only useful if you can identify the problem you want to solve. For example, “I’m not getting as much traffic to my website as I predicted, and I want more people to click-through to my landing page”.
- Identify the elements that need to be tested: Now, if your open rate is at around 35 % and you’re still seeing a very small click-through rate, the problem is possible not in the subject line or from name, but in the elements your contacts see when they open their email. Is your heading engaging? Are your CTAs easy to locate? Is your design appealing? Consider what elements can factor into your results and come up with ideas to shake things up a bit.
- Always think about your metrics: Your email stats will provide the information you’re looking for after your test, but it’s important to understand what results matter. If you’re testing different combinations of subject lines and from names, then the best performance indicator will be your open rates, whereas if you’re playing around with the color and text of your CTA, it’ll be click-through rate you’ll need to look at.
Awesome! So, how do I do this with Mailjet?
Fantastic question. Well, as you know, at Mailjet we don’t believe in testing just two email variations, which is why our A/B testing tool is not an A/B testing tool, but an A/X testing one. That means you can test up to 10 versions. Essentially, this means you can either come up with 10 really witty subject lines to perform a simple A/X test, or create different variations altering a number of elements within your message.
Mind-blown, we know…
But, we don’t stop there. You can also leverage Campaign Comparison to test different combinations across a period of time, like sending day and time. Does early morning on a weekday work better than midday during the weekend? Or is lunchtime on Mondays best?
Love it? Can’t wait to get started? Play around with our A/X testing tool and let us know how you’ve found your winning combination on Twitter, using the hashtag #EmailMarketing.