How to A/B test SMS campaigns

Last updated at:

You will learn

Learn how to A/B test SMS campaigns and what aspects to test in this article.

With A/B testing for Klaviyo SMS, you can gain insight into your audience and use that information to optimize your messages and send times. SMS A/B testing is currently only available for campaigns, but you can test your SMS flow messages using splits.

Before you begin

Klaviyo gives you the option to either test: 

  • Message content (e.g., the call-to-action, emojis, and images/GIFs)
  • Send times
Best practices for A/B testing SMS

Like with email, there are several best practices for A/B testing SMS:

  • Test 1 aspect at a time
  • Include a link and use the Klaviyo link shortener
  • Keep any images or GIFs under 600 KB
  • Test different types of SMS campaigns
  • Optimize your CTA and message text

You should also test different types of SMS campaigns; e.g., product promotion, newsletter, sale, etc. What performs well for one type of message may not perform well for another. Further, a campaign might yield different results than a flow message, so do not apply the results of an A/B test to every message you send.

A/B test SMS messages

Regardless of whether you want to test message content or send times, start by following these steps: 

  1. Click Campaign > Create Campaign > SMS.
  2. Choose your campaign name and recipients.
  3. Click Continue to Content.
  4. In the Message Content box, craft your message (you will not be able to create an A/B test until you add content).
  5. Click Create A/B Test to create a copy of your existing message (including any images, GIFs, personalization, etc.)
    SMS message creation screen, where you can create an A/B test
  6. Next, decide if you want to:
    1. A/B test the SMS content 
    2. A/B test SMS send times 

Test SMS content

  1. Click the dropdown menu for the variations.
  2. Choose the variation you want to change for the test.
  3. Change only one aspect from the original (e.g., if you add an image, do not change anything else about the message).
  4. Click Continue to Test Settings in the upper right.
  5. Review your A/B test settings (click the link to skip to that section).

If you are A/B testing content and would like to send your message based on each recipient’s timezone, set your test size to 100%. When scheduling your campaign, you’ll have the option to choose Recipient’s Local Timezone as the timezone for your send.

A campaign scheduling modal set to send based on recipients' local timezones

Test SMS send times

  1. Segment your audience by location or country.
  2. Choose which segment you want to A/B test.
    Note: you should only send the A/B test to one of these segments so that you don’t risk messaging someone too late at night or early in the morning.
  3. Keep the message content exactly the same.
  4. Select Continue to Test Settings in the upper right.
  5. In the A/B test settings page, click Switch to testing send times.
    The A/B test settings page, where you can choose to test sending times
  6. Choose the times you want to A/B test.
    SMS A/B test settings page when you choose to vary the sending times
  7. Configure the rest of the A/B test’s settings, which is discussed in the next section.

Adjust an A/B test’s settings (optional)

You can view the settings for your test and adjust:

  • The test's winning metric (click rate or placed order rate)
  • What percentage of people receive each variation
  • The test duration

Once a winning variation is selected, the winner will be sent to the rest of the audience.

A/B test settings page

Additional resources

Was this article helpful?
8 out of 13 found this helpful