How to A/B test campaign emails

You will learn

Learn how to set up and run a campaign A/B test, how to read the results of an A/B test, and a few use cases for A/B testing campaigns. Klaviyo's A/B testing feature for campaigns allows you to easily test different subject lines, message content, and send times so you can better understand what works best for your audience. 

How to A/B test campaign emails

1. Navigate to Campaigns > Create Campaign > Email.
2. Name your campaign.
3. Select the lists or segments you want to send to.
4. Click Continue to Content.
5. Input the subject line and, if desired, edit the preview text, sender name, and sender email address. 
6. Create the first version of your email, adding your copy, images, and links.
7. Click Save Content.
8. At the bottom of the page, click Create A/B Test. This will automatically create a second, identical variation of your campaign and bring you to the Campaign A/B Test page.
   
   
9. In each Variation Name field, add a descriptive variation name that references what you're A/B testing. 
   

Test content

On this page, edit one of the variations for your A/B test. It’s important for A/B tests to compare one factor at a time, so if you edit the subject line, do not change anything else. 

Clone a variation

If desired, you can add more variations by clicking the Clone button, although we recommend sticking to just 2 variations. 

Test send times

You can also test send times to determine when your audience wants to hear from you. To do so, click Switch to testing send times at the top of the page.

When testing send times, keep the content and subject lines of both variations exactly the same. 

If you don't know where to start, we recommend trying an evening time. Alternatively, if you qualify, you can use Smart Send Time instead. 

Configure A/B test settings

After creating your content, decide the size of your testing pool and the length of your testing period.

First, choose the winning metric, which is the main metric you want to see improve. Choose from: 

• Open rate
  Recommended when testing subject line, preview text, or sender name/email address.
• Click rate
  Recommended when testing email content, like button size or color.
• Placed order rate
  Only available for accounts with a Placed Order metric; recommended when testing email content, like whether displaying best sellers or new items leads to more conversions. 

Klaviyo recommends a test size based on your list size and a test duration based on the A/B test metric (i.e., opens, clicks, or placed orders). Use the slider bar to change the test size if desired. If the test size is less than 100%, you also have the option to adjust your test duration. 

For the example below, 20% of the campaign’s recipients will get variation A, and another 20% will get variation B. Depending on how many of these recipients open, click, or place an order (depending on the winning metric you selected), a winner will be chosen after 6 hours, and the rest of the recipients will receive the winning variation. 

After you review and adjust the test’s settings, click Continue to Review. 

What to A/B test 

You should only test one variable at a time. Testing multiple variables simultaneously can make it difficult (or impossible) to identify which variables contribute to improved performance. Not sure what to test? Use the ideas below as a starting point. 

Tests to improve your placed order rates

To see what drives your recipients to place orders, try testing any of the following in your messages: 

• Social proof (e.g., including positive reviews or customer social media posts in your messages)
• Send time, day of week, or weekends versus weekdays
• Call to action (CTA) placement (e.g., near the top of your email or lower down)

Tests to improve your click rates

To see what drives your recipients to click on links in your messages, try testing any of the following: 

• CTA appearance (e.g., the font, color, or size of your button or text)
• CTA text (e.g., "Shop now" versus "Check it out!")
• Template organization (e.g., how you arrange the images and text in your message)
• Images versus GIFs

Tests to improve your open rates 

To identify what prompts your subscribers to open your emails, try testing any of the following: 

• Subject line tests: 
• Statements versus questions (e.g., “You’ve gotta see our latest drop” versus “Want to see our latest drop?”)
• Subject line length
• Subject line personalization (e.g., including the recipient’s first name in the subject line using the {{ first_name|default:’Friend’ }} tag)
• Emojis in subject lines
• Preview text (e.g., including versus not including preview text)
• From name (e.g., your company name compared to a more personal From name, like “Elise at Klaviyo”)
• Send time, day of week, or weekends versus weekdays

Review A/B test results and select a winner (optional)

Navigate to the campaign you are testing to view its progress and see which version has performed better. If you'd like to end an ongoing A/B test:

1. Navigate to the A/B Test Results tab of an ongoing campaign.
2. Click the additional options menu next to the version you'd like to choose.
3. Click Choose as Winner.

When you manually select a winner, anyone who hasn't yet received the campaign will receive the winning variation immediately. If your A/B test size was 100% of recipients, we will continue collecting data throughout the conversion window for the message, so the winner may change as new data comes in. For more information, check out How to review your email A/B test results for campaigns.

Additional resources

• Best practices for A/B testing
• How to A/B test a flow email
• How to A/B test SMS campaigns
• How to resend a campaign that exceeded account sending limits