You will learn
Learn about what you can test in your flows using Klaviyo's A/B testing feature.
Flows send automatically to customers, making them an instrumental part of your communication strategy. You can simply set up a flow, turn it live, and then leave it to send; however, because flows represent your brand at key touchpoints in the customer journey, it's important to test and optimize flows.
Note that you should only test one variable at a time. If you test more than, the results may be skewed.
With the release of iOS15, macOS Monterey, iPadOS 15, and WatchOS 8, Apple Mail Privacy Protection (MPP) changed the way that we receive open rate data on your emails by prefetching our tracking pixel. With this change, it’s important to understand that open rates will be inflated.
If you are triggering flows off of opens themselves, we suggest creating a custom report that includes an MPP property to review these affected opens. You can also identify these opens in your individual subscriber segments.
For complete information on MPP opens, visit our iOS 15: How to Prepare for Apple’s Changes guide.
When should your flow messages send out? To answer this question, you can use a conditional split to find the optimal timing for a flow message. Drag a conditional split before the time delay you want to test. Then, configure your split to be based on a 50% random sample. This will allow you to run an A/B test on the timing of one of your messages.
Bear in mind that if you change the weighting of the random sample to something other than 50%, that will be the percentage of people who flow down the YES path.
Next, drag a time delay to the NO branch of the flow. Set this to be the other timeframe that you would like to test.
Last, clone all of the components on the YES branch and drag the clones over to the NO branch, so that everything is identical except the timing of the message. This allows you to isolate this as the variable that you're testing.
After you set the cloned messages live, monitor their conversion rate of the messages to determine which time delay is performing best. Then, you can test another time delay or delete the split and only keep the winning time delay.
You can repeat this process with as many messages in a flow as you would like, but remember to A/B test them one at a time so that you can isolate this as the only variable you're testing.
Number of messages
You may want to test how the number of messages in your flow affects your conversion rate. For example, maybe you want to test adding another message to the flow. When analyzing this message, pay special attention not only to its conversion rate but also the open and click rates. If the additional message has poor open and click performance, it could be doing more harm than good for your deliverability, even if it’s producing revenue.
To test the number of messages in your flow, drag in a conditional split below the trigger. Base the split on a random sample and select the weighting you would like to assign to the control branch. The test branch will be the NO branch or the remainder of people who enter the flow.
Subject lines (email only)
If the content of this section does not match the A/B testing experience you see in your account, this is because your account has access to the updated A/B testing experience. Learn more from our article on How to A/B test a flow email using the updated A/B testing experience.
Subject lines are directly tied to your open rates.
- To test a new subject line, first, click on the email that you would like to test.
- Click Create A/B Test in the upper corner of the lefthand sidebar.
- You will see sections for 2 different variations of the email. You can change the Subject line field for each variation.
Click rates, on the other hand, are primarily affected by the content and, for emails, the layout. You can A/B test your message content by changing the body of the email or text, but nothing else (e.g., subject lines). You can measure your results in the same way you would measure them for a subject line A/B test, but instead of looking at open rate, pay special attention to the message's click rate.
Images and GIFs
For emails, changing the images or GIFs is part of testing the overall content, but that's not the case for your text messages. The question of whether or not to add an image or GIF is much bigger for text message marketing. A/B testing can help you decide if you should use an SMS or MMS message and for MMS messages, if it should be an image or GIF.
Learn more about A/B testing: