I have a notification that my form that has a time delay A/B test is calculating form submit rate differently. Why is that?
If you’re testing a time delay on your form, your form submit rate will be calculated by looking at the total conversions as a percentage of the total people eligible to see your form, rather than the number of views the form had.
Say, for instance, your A/B test is evenly weighted and has the message, “Need help browsing?” In the test, one form shows immediately upon someone visiting your site, and the other form loads after a minute. While the same number of people will be queued up to see both forms, more people will see the form that shows immediately. While there will not be as many people who see the form with the delay, the form offers help to browsers who may be confused, and thus may still have a greater form submit rate. To test if the form delay leads to a higher or lower submission rate, the form submit rate will not be the amount of submits as a percentage of the number of views, but rather the amount of submits as a percentage of the number of people who could have seen the form.