You will learn
Learn what you can find on the A/B test results page and what each of these indicators mean.
A/B testing campaigns can provide valuable insight into your audience. Klaviyo makes it simple to determine what changes you should make to your sending strategy. Not only will you know the winning variation for an A/B test, but also other important factors, like the win confidence percentage and winning metric details.
A/B test results
To see the results of an A/B test:
- Navigate to the campaign you are testing
- Select the A/B Test Results tab of the main campaign report
Test overview
In the Test Overview section, you’ll see a snapshot of the test results, including:
- Winning variation
Once a test is complete, the variation that won - Win confidence
How likely it is that the winning variation truly had the highest open, click, or placed order rate after taking random chance into account - Statistical significance
Whether the winning variation won by a sufficient margin to be considered statistically significant - Status
Whether the A/B test is ongoing (i.e., collecting analytics) or complete - Winning metric
Which metric was used to determine the winning variation (i.e., open rate, click rate, or placed order rate) - Test size
The percentage of message recipients who were part of the testing pool
Understanding win confidence and statistical significance
Say, for instance, you want to see if adding emojis to your subject line impacts your open rate with your newsletter list. If you only send the test to 2 subscribers and the emojis have a better open rate, you can’t say conclusively that the rest of your newsletter list will feel the same way. If instead you tested with 100 people in your newsletter list and found that emojis have a substantially higher open rate, you would feel much more confident that using emojis in your subject lines is better for your audience.
Under the win confidence, you’ll see whether or not the variation shown is statistically significant. If it is deemed significant, this means that the variation has a high win probability (i.e., 90% or higher). A test might also be promising, inconclusive, or not statistically significant. Learn more about statistical significance in Klaviyo.