A/B Test a Flow Email

read

Overview

Klaviyo's A/B testing feature for flows allows you to easily test different subject lines and even completely different email content to help you refine your messaging. 

Flow A/B testing is the process of:

  1. Creating a single flow email and scheduling this email to send after a certain number of hours or days
  2. Configuring 2 or more email variations for this single flow email -- adjusting the subject line or core email content for each variation
  3. Selecting the weight of each variation, to establish the percentage of recipients that will get each email
  4. Observing the performance of each variation over time, to determine which should be considered the "winner"

Once you determine that one variation is out-performing the others, you can delete the other variations so that 100% of your recipients will receive the tested winner. 

Set Up a Flow Email A/B Test

To set up A/B testing for an email in any flow, select the email card and click Add Variation in the upper righthand corner of the sidebar. You will be taken directly to a Manage Variations window. Here, you can edit the subject line and/or content of each variation by clicking Edit Variation.

To add another variation, click Add Variation in the top menu or alternatively click on the Actions dropdown for an existing variation and select Duplicate Variation.

Lastly, you will want to choose the weight of each variation - this will establish the percentage of recipients that will get each one. It's common practice to weight each variation equally, which you can accomplish quickly by clicking Equalize Weights.

2018-01-12_22-51-35.gif

If you've just configured a new email and are in the Message Content view, you can quickly add a variation without returning to the main flow canvas by clicking the Add Variation link below your configured message. Clicking this link will bring you directly to the Manage Variations window where you can edit the subject line and/or content of each variation.

AddVariation.png 

After setting your flow email with A/B testing live, you can monitor the performance of each variation across the following metrics:

  • Recipients: How many people received the email
  • Opens: What percentage/number of people opened the email 
  • Clicks: What percentage/number of people clicked through the email
  • Placed Order: 
    • Total of recipients who placed an order
    • Total dollar amount that was made from each email

Flow Email A/B Testing Examples

A few examples of how Klaviyo customers use A/B testing:

  • Experimenting with new subject lines or types of emails in a very low-risk way. Because A/B tests allow you to send unique variations to a limited subset of people, this is a great way to try something new and get real data to measure how well the idea worked (rather than having to debate the anticipated outcome without metrics to back up your assumptions).
  • Understanding the right discount to offer and whether they are necessary. Because you can easily test completely different email content, A/B tests are a great way to figure out if you really need to offer 20% off or if you can instead get away with 10% or no offer at all.
  • Figuring out whether plain text emails perform better than image-heavy HTML emails. Many of our customers test out both very short personal emails as well as traditional emails with images and complex layouts. Both work well for different purposes, but A/B testing will give you actual numbers to analyze when considering which approach is best.
Was this article helpful?
0 out of 0 found this helpful