Marketing decisions shouldn’t be left to guesswork. What works for your audience and what doesn’t, should be put to the test and the results of those should define the future work. Here’s where AB testing comes into play. Split testing or AB testing is a term we use in marketing to describe the process of putting to the test two different versions of something (in this case, email campaigns) and see which is the best performing one to find conclusive results of what works better for our target audience. This is important in the email marketing world because, this way, we have reliable insights in order to base our next campaigns and strategies on.
So, let’s dive deep into the what, why, and how of AB testing email campaigns. Get ready to know all about:
- How to AB test email campaigns
- How to understand the results of your split tests
- Plus, 22 design, copy, and strategical AB tests to try on your email campaigns.
1. How to AB test email campaigns
Before starting your test, you need to keep in mind five important things:
a. What to measure?
What do you want to measure? Do you want to know if the thing you’re changing affects open rates or click rates? Subject lines and preview headers will definitely affect the open rate of your email. While any element inside the email per se will affect your click rate directly.
b. Isolating test element
When AB testing email campaigns, you want to choose a unique element to isolate it to be the only difference between two otherwise identical emails. This is so you can attribute the better performance of one of the two email campaigns to that only change.
c. Test groups
To split test your email campaign, you need to send each version of your email to a different group of people. So, if you have two email versions, you’ll also have two groups. If you have three versions, you will need three groups.
d. Test duration
You can choose to send your two emails versions to say 40% of your segment (that 40% will be divided evenly into two groups) and test the variations for a specific number of hours, and after that time ends up, the email with the higher open rate or click rate, depending on the metric you shoes to measure, will be sent to the remaining 60% of the segment. The larger the test groups’ size, the more accurate results you will get.
e. Entire segment test
You can also choose to divide your entire segment into two groups (each one with 50% of your segment). If so, you won’t need to set up a test time, but the platform will evaluate the performance of the emails based on the interaction it has at all times.
Now, every platform is different, but, the process of AB testing email campaigns is always easy. Let’s see how to do it with our good old friend, Klaviyo.
1. Create a new campaign, set the segments, and continue to the content:
2. Prepare your first version: subject line, preview header, and template:
3. Click on the AB test email campaign option:
4. Prepare your second version:
5. Select the metric you want to measure, choose the group size according to your strategy (and Klaviyo’s recommendations) and, finally, choose the duration of the test.
6. Continue to review and schedule your campaign:
How to understand the results of your AB tests
When you AB test an email campaign, you’ll find an AB test results section in the campaign’s results overview. There you’ll find the following:
a. The winner variation and Klaviyo’s confidence in the test. If there is not an important difference between both versions, Klaviyo will let you know the test was “statistically insignificant”.
b. The status of the test (has it been completed or not yet)
c. What was the winning metric you selected for the test and with how much percentage difference did the winner version outperform the other version.
d. The test size. If you tested the two versions with 40% or 100% of the segment.
e. You’ll find the expanded results with the information for each version: deliveries, open rate, click rate, and placed order.
These are all the elements you should check out every time you do a test. We also suggest creating a sheet where you can add up the results of all your email AB tests so you can have a look into the bigger picture of your AB testing strategy. This is essential since you need to perform the same AB test multiple times to get conclusive results. That is to say, you need to repeat the same type of test until you see a clear pattern: version A wins most of the time, version B wins most of the time, or you always get inconclusive results because the element your testing actually doesn’t make a difference. Keep a record of your results and test systematically to update your insights regularly.
22 AB testing ideas for your email campaigns
Here you’ll find 22 creative and essential split testing ideas for your email campaigns, plus our curated blog recommendations and best practices for each element of your email campaign, so you can perform quality AB tests that will give you quality insights about your audience.
Design
1. Main banner with CTA button vs. Main banner with no CTA button
2. Animated CTA button vs. Non-animated/static CTA button
3. Image-Heavy vs. Text-heavy design. It’s always important to have a good text to image ratio in your email campaigns. See what works best for your audience and your product.
4. Plain text vs. Branded email design
5. Design style (Minimalistic vs. Art Nouveau vs. Grunge)
6. Header menu vs. No header menu. See if having a menu at the top with CTAs to your different collections helps increase your click rates.
7. CTA position (left, center, right). Recent studies show that people tend to click the right call to actions more than the left ones. Put this to the test with your own audience.
Copy
8. Creative CTA vs. Straight-forward CTA.
9. Long copy email vs. short copy email. Check these curated tips to make sure your copywriting is effective and engaging whether it is long or short.
10. Subject lines tone and angle. Try different angles like humorous vs. direct or pain-point focused vs. unique selling point focused.
11. Subject lines with emojis vs. Subject lines with no emojis
12. Subject lines lengths (long vs. short). Need inspiration? Check these 200 great email subject lines for flows and campaigns.
13. Subject lines mentioning the offer vs. no offer mention
14. Using personalized greetings vs. Generic greetings. Maybe your audience prefers you to address them by name, or maybe they are fine with generic greetings. Only AB testing will tell.
Product Showcase
15. Using dynamic and personalized product showcases vs. curating your own product showcase. You can use data feeds to showcase products according to what your subscriber has browsed and purchased (this will change for every profile) or you can pick your favorite products and showcase the same ones to all your subscribers.
16. AB test the collections you use for your dynamic product blocks. You can power your dynamic product showcases with different collections Which one will work best? New arrivals or best sellers? Test it out!
17. Showcasing the price vs. not showcasing it
18. Writing the product name vs. not writing it
19. Including a description for each product vs. no product description
20. Using lifestyle pictures vs. flat lay product pictures
STRATEGY
21. Send times and days. Your audience can go online at certain times more often than others. Same thing with the days. Check your Google Analytics to see the times and days your audience is usually browsing your website to test those times out with your email campaigns.
22. Offers. Get ready for Black Friday in advance by AB testing different types and amounts of discounts and see which one makes your audience most likely to buy from you
As you can see, there are many elements you can AB test in your email campaigns. So, try the ones that make sense to you and your brand. Once you have these insights about your audience, use them to plan an effective email content strategy that boosts sales. And, if you need help with your email marketing strategy, don’t hesitate to book a free call with us.
