How to A/B Test SMS Campaigns

read
Last updated at:

Overview

Learn how to A/B test SMS campaigns and what aspects to test in this article.

With A/B testing for Klaviyo SMS, you can gain insight into your audience and use that information to optimize your messages and send times. SMS A/B testing is currently only available for campaigns, but you can test your SMS flow messages using splits.

A/B Testing SMS Messages

With SMS, you can test either the message content (including the CTA, emojis, and images/GIFs) or the send times. The sections below detail both options.

In either case, click Campaign > Create Campaign > SMS.

Choose your campaign name and recipients, then click Continue to Content.

In the Message Content box, craft your message. You will not be able to create an A/B test until you input content.

Click Create A/B Test; this will automatically create a copy of your existing message (including any images, GIFs, personalization, etc.)

A_B_test_SMS.gif

Testing SMS Content

In the dropdown menu, choose which variation you want to change for the test.

Remember to change only one aspect from the original (e.g., if you add an image, do not change anything else about the message).

SMS_Content_for_A_B_testing.png

Click Continue to Test Settings in the upper right. You’ll then be taken to the A/B test settings page, which is discussed in this section.

Testing SMS Send Times

Before A/B testing send times, segment your audience by location or country. Only send an A/B test to those within one of these segments so that you don’t risk messaging someone too late at night or early in the morning.

When testing SMS send times, it’s important to keep the message content exactly the same. Without changing either variation, select Continue to Test Settings in the upper right.

In the A/B test settings page, click Switch to testing send times.

Next, choose the times you want to A/B test.

GIF_of_A_B_testing_send_times.gif

Lastly, you’ll just need to configure the rest of the A/B test’s settings, which is discussed in the next section.

Adjusting an A/B Test’s Settings

You can view the settings for your test. The winning metric is always the click rate for SMS, but you can adjust:

  • What percentage of people receive each variation
  • The test duration

Content_Test_Settings_SMS_A_B_testing.png

Once a winning variation is selected, the winner will be sent to the rest of the audience.

Changing_the_test_size_for_an_SMS_A_B_test.png

What to A/B Test in an SMS

Now that you know how to perform an A/B test, it’s time to decide what to test. Like with email, there are several best practices for A/B testing SMS:

  • Test one aspect at a time
  • Include a link and use the Klaviyo link shortener
  • Keep any images or GIFs under 600 KB
  • Test different types of SMS campaigns
  • Optimize your CTA and message text

You should also test different types of SMS campaigns; e.g., product promotion, newsletter, sale, etc. What performs well for one type of message may not perform well for another. Further, a campaign might yield different results than a flow message, so do not apply the results of an A/B test to every message you send.

Optimizing the Message Text

The text for a message includes many different aspects. A/B testing the text helps you understand if your audience responds positively or negatively to acronyms, text-speak and emojis.

Another aspect is the link placement. Do your subscribers respond better if it’s place in-line, on a new line, or in the middle of a message?

Further, A/B testing can help you craft better call-to-actions, determine whether to use an organizational prefix or work your brand into the message, and figure out how long to make the message and the best tone to use.

Including Media: SMS vs. MMS

The difference between an SMS and MMS message is: MMS messages include some type of media (an image or GIF), whereas SMS messages don’t.

It may be that your audience prefers one message type over the other. To test this, you must keep the text exactly the same. Only then can you see if it’s worth including media in your messages.

As a follow-up to this test, you can see if images or GIFs tend to perform better. When evaluating this, show the same subject in both the image and GIF.

Additional Resources

Best Practices for A/B Testing

Statistical Significance

How to A/B Test via Flow Branches

x
Was this article helpful?
2 out of 2 found this helpful