What to A/B Test in Your Flows

Last updated at:


Flows send automatically to customers, making them an instrumental part of your communication strategy. You can simply set up a flow, turn it live, and then leave it to send; however, because flows represent your brand at key touchpoints in the customer journey, it's important to test and optimize flows.

Note that you should only test one variable at a time. If you test more than, the results may be skewed. 

This article shows you what you can test in your flows.

9/20/21 Update: with the upcoming release of iOS 15, macOS Monterey, iPadOS 15, and WatchOS 8, Apple will change the way that we receive open rate data on your emails by prefetching our tracking pixel. With this change, it’s important to understand that open rates will be inflated. 

Thus, if you are triggering any flows off of opens themselves, we suggest moving to expand that trigger definition to include other engagement criteria like clicks or product views. It may also be useful to look at the number of iOS-specific users within your data to understand how your open rates will change.

For complete information on ways to potentially mitigate these Apple open changes, visit our iOS 15: How to Prepare for Apple’s Changes guide.


When should your flow messages send out? To answer this question, you can use a conditional split to find the optimal timing for a flow message. Drag a conditional split before the time delay you want to test. Then, configure your split to be based on a 50% random sample. This will allow you to run an A/B test on the timing of one of your messages. 

Bear in mind that if you change the weighting of the random sample to something other than 50%, that will be the percentage of people who flow down the YES path.


Next, drag a time delay to the NO branch of the flow. Set this to be the other timeframe that you would like to test.


Last, clone all of the components on the YES branch and drag the clones over to the NO branch, so that everything is identical except the timing of the message. This allows you to isolate this as the variable that you're testing.

After you set the cloned messages live, monitor their conversion rate of the messages to determine which time delay is performing best. Then, you can test another time delay or delete the split and only keep the winning time delay.

You can repeat this process with as many messages in a flow as you would like, but remember to A/B test them one at a time so that you can isolate this as the only variable you're testing.

Number of Messages

You may want to test how the number of messages in your flow affects your conversion rate. For example, maybe you want to test adding another message to the flow. When analyzing this message, pay special attention not only to its conversion rate but also the open and click rates. If the additional message has poor open and click performance, it could be doing more harm than good for your deliverability, even if it’s producing revenue. 

To test the number of messages in your flow, drag in a conditional split below the trigger. Base the split on a random sample and select the weighting you would like to assign to the control branch. The test branch will be the NO branch or the remainder of people who enter the flow.


Subject Lines (Email Only)

Subject lines are directly tied to your open rates. To test a new subject line, first, click on the email that you would like to test. Then, click Add Variation in the lefthand panel.



Click rates, on the other hand, are primarily affected by the content and, for emails, the layout. You can A/B test your message content by changing the body of the email or text, but nothing else (e.g., subject lines). You can measure your results in the same way you would measure them for a subject line A/B test, but instead of looking at open rate, pay special attention to the message's click rate.

Images and GIFs

For emails, changing the images or GIFs is part of testing the overall content, but that's not the case for your text messages. The question of whether or not to add an image or GIF is much bigger for text message marketing. A/B testing can help you decide if you should use an SMS or MMS message and for MMS messages, if it should be an image or GIF. 


Additional Resources

Learn more about A/B testing: 

Was this article helpful?
8 out of 9 found this helpful