Signup forms are one of the most important tools for acquiring customers — but how do you know if your signup form is the best that you can create? A/B testing signup forms allows you to test different elements of your signup form to ensure that it resonates well with viewers.
Creating an A/B Test
Creating a Test
There are three different ways to create an A/B test for your signup form:
- From the left-side menu on the signup form page
- By clicking into the analytics for a live form
- From inside of the form builder
No matter where you enter from, select Create A/B Test to get started.
Next, name your test something descriptive. If you’re testing colors of the buttons on your newsletter popup, you could name the test Newsletter - Color of CTA. Not sure what to test? Check out our article on A/B testing best practices for more ideas.
After you’ve named your A/B test, give your signup form variations different names by selecting the pencil icon by your form’s current name. By default, it will come with the name Copy # of Name of Form. With the different color CTA example from before, you might have your forms named Newsletter - Blue and Newsletter - Gray.
Here, you can also add up to seven variations to your A/B test. Keep in mind, the more variations you add, the longer you’ll need to run your test. An ideal A/B test would have between two to four variations. In the example below, there are three variations: the original form and the two additional tests. After you’ve configured your test, select Create Test.
Configuring the Variations
After you’ve created your test, you’ll be directed back to the Signup Form builder. You will be able to toggle between the different variations of your form by selecting the name of your form. From this menu, you can also choose to change the name of the variation, create an additional variation, or delete the variation.
Once you’re done configuring your form, select Continue to Test Settings.
Configuring A/B Test Settings
You can reach the test settings page in three different ways. You can click the Continue to Test Settings button from the signup form builder, select a form in draft mode with a draft test from the signup form landing page, or select Edit Test from the A/B test results page. The customizable settings of your test will appear here. You can continue to customize the title, notes, and variations of the test from the top of the settings menu.
Additionally, if you select Preview, you’ll be able to see the settings of each variation of the test. You will not be able to edit the content of a form when an A/B test is in progress. If you want to edit the form, you will need to stop the test.
From this section, you can determine how much traffic goes to the variations of your form. By default, each form will appear to an equal amount of viewers. As your test runs, if one of your variations is outperforming the rest, Klaviyo will direct traffic to the better performing form while the test is still running.
If you toggle to select Weight - Manual, you can specify how much traffic you want to go to each variation of the form. The amounts need to add up to 100%.
Similar to the Traffic Selection section, you have two options in determining how to choose the winner of the test. You can end the test automatically or manually.
If you end your test automatically, it will end when there is one variation that outperforms the other(s) at a statistically significant level or when it reaches a particular date. You can also select both boxes and the test will end when one of the conditions are met. For example, if you have both conditions selected and your form will only run for a month, if the form has statistically significant results after a week, the test will end.
If you end your test when the results are statistically significant alone, Klaviyo’s data science model will do the math for you and pick a winner.
If you choose to send the test on a particular date, the test will end at midnight coordinated universal time (UTC).
Regardless of which automatic selection you pick, after the test has finished, the form that wins will be live on your site.
If you choose to end your test manually, you will choose both the winner and the date that the test ends. To stop your test, select Choose Winner from the A/B test settings page of a live test. After you stop your test, the test you select as a winner will be live on your site.
After you’ve adjusted the test settings, select Publish A/B Test. Before your test is underway, Klaviyo will check if there is an issue publishing forms to your site and if your form has all of the content it needs (e.g., consent language, CTA, submit fields, etc.).
Review Test Results
While the test is running or after you’ve completed your A/B test, you can see the data in your account by clicking on the form. In the overview page, you can see how your form is performing overall. Click to the A/B Test Results tab to dive into the analytics of your A/B test specifically.
Edit A/B Test
You can edit your A/B test from the A/B Test Results page. Note that while you can change the test settings, you won’t be able to change the test content. To edit the content, you’ll have to end the test and start it again.
From this button, you’ll be able to manually stop the test and choose a winner at any time. If you select Choose Winner, you’ll have the option to end the test and set your form to live or draft mode.
This button will only appear on tests that are completed. Here, you will be able to edit the test name and notes.
In the test overview panel, you’ll get a snapshot of how your test is performing.
The win probability denotes the likelihood that the current variation that is winning will be the statistically significant winner. As your test gains more viewers and is live for a longer period of time, you should see this number increase. You’ll know when the test results become statistically significant from the label below the win probability. If you’re viewing a completed test, there will also be a reason stating why the test ended (e.g., manually ended by choosing a winner).
Test Name, Status, Start Date
The next three sections denote the name of the test, the status of the test (e.g., live or completed), and the date the test began. If the test is completed, you’ll also see the date the winner was selected.
Below, you’ll see the current weight of traffic directed to each form. If you have manually selected how much traffic goes to each form, these numbers will be consistent with the settings you selected when you started the test. If you’re using Klaviyo’s data science model, you will see the weight of traffic changing over time, giving more traffic to the form that is winning and letting you gain more subscribers as you run your test.
Form Submit Rate Graph
Next to the test overview is a plot of your variations. It shows the form submit rate by variation over the duration of the A/B test. If you’re interested in a particular date, hover over data points for more information.
Below the graph, you’ll see the raw data from your test. Next to each variation, you can see a preview of each form, the number of views, the number of people who have submitted your form, and the form submit rate for each variation.
A/B Testing Ideas
While you can test any idea that you may have for your forms, some common ideas for testing are:
- Type of form
New to A/B testing? Head to our article for best practices on A/B testing.
- When I create an A/B test for my form, will my form go to draft mode?
- I selected both boxes for the Winner Selection part of my A/B test. Will the test end when both or either one of the conditions are met?
- Can I edit the test settings for my form A/B test after it's already running?
- If someone doesn’t engage with a form that is a part of an A/B test, when the form shows to them a second time, will they see a different version of the form?
- I had a typo in my form A/B test, so I published a new one. How can I permanently delete the test with a typo?
- One of my form variations is performing very poorly and I want to update the amount of traffic going to the form. Can I do that?
- I want to track who sees each form variation of the A/B test. How can I do that?