How to migrate to the updated flows A/B testing experience

Last updated at:

You will learn

Learn how to update your existing flows’ A/B tests to Klaviyo’s updated and improved A/B testing experience. You'll be able to create more comprehensive, data-backed A/B tests with automatic winner selection and easier access to your historical testing data.

Any new A/B tests you create will automatically use the updated testing experience, while your existing tests may still be using the old A/B testing functionality. This article will walk you through the steps needed to move those existing tests to the updated A/B testing experience. 

For details on how to set up new A/B tests using the updated version, please see our article on how to A/B test a flow email using the new A/B testing experience.

Updated A/B testing features overview 

Improved performance analysis: 

  • Instead of manually choosing a winner, tests can automatically select a winner based on reaching statistical significance or by setting an end date
  • Quickly compare the performance and win probability between variations

Improved efficiency: 

  • Easily access test details
  • Manual processes have been reduced in order to evaluate success and declare winners

Increased visibility: 

  • View A/B test historical results so important information is not lost
  • The ability to run multiple tests for the same message at different times

Update existing tests in flows to the updated experience

When you navigate to the Flows tab, you will see a message informing you about the updated A/B testing experience.

A message will appear above the flows list with information about the updated A/B testing experience.

To update your existing tests, follow these steps:

  1. To the right of this updated A/B testing message, click the Update button.

  2. A modal will appear, informing you how many A/B tests in your account are still running and are using the old version of A/B testing. If you’d like, you can manually end existing tests – choosing a winner by changing the percentage of one of one of the variations to 100%.

    Otherwise, you will be presented with 2 options to update your existing tests that are still running:
    A modal will present you with 2 options to update existing tests.
    1. End statistically significant tests and restart all other tests (recommended option): existing tests that are statistically significant (at least 1000 people have received each variation and the win probability is at least 90%) will end while all other tests will be restarted using the updated A/B testing version. This is the recommended option.
    2. End all tests and choose a winner based on click rate: all existing tests will end with a winner chosen based on the variation with the highest click rate.
  3. Choose your preferred option then click the Update Now button in the bottom right of the modal.

  4. Once you have chosen an option, another modal will appear asking you to confirm your selection. Click the Update Now button to confirm.

    Updating your existing A/B tests to the updated experience is a permanent change across your entire account and cannot be undone.

    A modal will appear asking you to confirm that you want to update your existing tests.
  5. Now that you have successfully migrated to the updated A/B testing version, you can follow the steps in our guide on how to How to A/B test a flow email using the updated A/B testing experience.

Additional resources

Was this article helpful?
3 out of 4 found this helpful