How to review email A/B test results for flows

read
Last updated at:

You will learn

Learn how to review A/B test results for flow messages and what each of the indicators on the results page mean.

A/B testing flow messages can provide valuable insight into your audience. Klaviyo makes determining what changes you should make to your sending strategy simple. Not only will you know the winning variation for an A/B test, but also other important factors, such as winning metric details.

View the A/B test results page

The option to view test results will only appear for tests that have been published - this includes tests that are currently running and those that have completed. To learn more about creating and publishing a test, please see our article on how to A/B test a flow email.

To view the results of a test while it is running or after completion, follow these steps:

  1. Click an email within the flow builder that has a published A/B test
  2. In the left sidebar, click View A/B Test
    The 'View A/B Test' button found in the left sidebar of the flow builder
  3. The results page will automatically show the results for the current or most recent A/B test. If you want to view the results for previous tests, click the name of the test in the top left of the A/B Test Results page and choose the name of a previous test to view its results.
    The names of past A/B tests found on the A/B test results page

The results page is broken into two parts:

  1. Test Overview
  2. Variations

The A/B test results page with the overview and variations sections

Test Overview

In the Test Overview section, you’ll see a snapshot of the test results.

The following sections will appear for currently running tests:

  • Win Probability
    Which variation is likely to win and a percentage of how likely it is to winning
  • Winning Metric
    Configured winning metric (click rate or open rate)
  • Test Name
    Name of the test you are viewing results for
  • Status
    Whether the test is running or completed
  • Start Date
    Date and time the test started in your account’s configured timezone
  • Current Weights
    List of variations and the weight percentage of each

The overview section for a currently running test

The following sections will show for completed tests:

  • Winner
    Which variation won based on the winning metric
  • Test Name
    Name of the test you are viewing results for
  • Status
    Whether the test is running or completed
  • Start Date
    Date and time the test started in your account’s configured timezone
  • End Date
    Date and time the test completed in your account’s configured timezone

The overview section for a completed test

Win Probability/Winner

In this section, you’ll see the variation currently in the lead if the test is still running or the winning variation if the A/B test is finished.

Win probability for a currently running test

Win probability for a completed test

Winning Metric

The winning metric, which is the metric you choose when you start an A/B test, is shown here. This section shows the following information:

  • The name of the winning metric – either Click Rate or Open Rate depending on your test’s configuration
  • Whether or not the results are statistically significant
  • A percentage of how much higher the winning metric is for the winning variation compared to other variations – this will not appear if a winner was manually selected

Winning Metric subsection with a Click Rate winning metric

Status

This section will show the status for the test, either Live for a currently running test or Completed for a test that has chosen a winner.

Start Date/End Date

The start date and time of the test will display according to your account’s timezone. If the test has completed, the end date and time will also appear.

Start Date and End Date of a completed test

Current Weights

This section will only appear if the test is still running, and will show what percentage of randomly selected recipients received each variation. You can click on the name of the variation to view a preview of the message.

Current Weights section showing 2 different variations

Variations

In the Variations section, you’ll see a chart of the winning metric over time per variation as well as a card with each variation.

Chart displaying Click Rate over time

The following metrics are tracked for each variation:

  • Recipients
    How many people received the email
  • Opens
    What percentage/number of people opened the email 
  • Clicks
    What percentage/number of people clicked through the email
  • Conversion Metric
    Chosen conversion metric and number of recipients who converted via that metric

The default conversion metric for most ecommerce integrations is Placed Order. If you use this metric, the card will show the placed order rate and the number of recipients who placed an order.

When you hover your cursor over each variation, you will see the option to preview the variation as well as save it as a template.

Variations sections showing metrics for 2 variations

Additional resources

Check out this strategy guide: Best Practices for A/B Testing

Learn about A/B tests you can run:

x
Was this article helpful?
1 out of 1 found this helpful