How to review email A/B test results for flows

read
Last updated at:

You will learn

Learn how to review A/B test results for flow messages and what each of the indicators on the results page mean.

A/B testing flow messages can provide valuable insight into your audience. Klaviyo makes determining what changes you should make to your sending strategy simple. Not only will you know the winning variation for an A/B test, but also other important factors, such as winning metric details.

View the A/B test results page

The option to view test results will only appear for tests that have been published - this includes tests that are currently running and those that have completed. To learn more about creating and publishing a test, please see our article on how to A/B test a flow email.

To view the results of a test while it is running or after completion, follow these steps:

  1. Click an email within the flow builder that has a published A/B test.
  2. In the left sidebar, click View A/B Test.The 'View A/B Test' button found in the left sidebar of the flow builder
  3. The results page will automatically show the results for the current or most recent A/B test. If you want to view the results for previous tests, click the name of the test in the top left of the A/B Test Results page and choose the name of a previous test to view its results.
    The names of past A/B tests found on the A/B test results page

The results page is broken into two parts:

  1. Test Overview
  2. Variations

The A/B test results page with the overview and variations sections

Test Overview

In the Test Overview section, you’ll see a snapshot of the test results.

The following sections will appear for currently running tests:

  • Win Probability
    Which variation is likely to win and the percent likelihood that a given variation has of winning
  • Winning Metric
    The winning metric configured during test setup (either Click Rate or Open Rate), whether or not the results are statistically significant, and a percentage of how much higher the winning metric is for the winning variation compared to other variations – this will not appear if a winner was manually selected
  • Test Name
    Name of the test you are viewing results for
  • Status
    Whether the test is running or completed
  • Start Date
    Date and time the test started in your account’s configured timezone
  • Current Weights
    List of variations and what percentage of randomly selected recipients received each variation

The overview section for a currently running test

The following sections will show for completed tests:

  • Winner
    Which variation won based on the winning metric
  • Test Name
    Name of the test you are viewing results for
  • Status
    Whether the test is running or completed
  • Start Date
    Date and time the test started in your account’s configured timezone
  • End Date
    Date and time the test completed in your account’s configured timezone

The overview section for a completed test

Variations

In the Variations section, you’ll see a chart of the winning metric over time per variation as well as a card with each variation.

Chart displaying Click Rate over time

The following metrics are tracked for each variation:

  • Recipients
    How many people received the email
  • Opens
    What percentage/number of people opened the email 
  • Clicks
    What percentage/number of people clicked through the email
  • Conversion Metric
    Chosen conversion metric and number of recipients who converted via that metric

The default conversion metric for most ecommerce integrations is Placed Order. If you use this metric, the card will show the placed order rate and the number of recipients who placed an order.

When you hover your cursor over each variation, you will see the option to preview the variation as well as save it as a template.

Variations sections showing metrics for 2 variations

Additional resources

Learn how statistical significance is determined in flows.

Check out our strategy guide on best practices for A/B testing.

Learn about A/B tests you can run:

x
Was this article helpful?
3 out of 4 found this helpful