- How A/B Testing Works
- Create and Edit an A/B Test
- Start Your A/B Test
- Add Notes
- Ending a Test— Analyzing Results— Choose a Winner— End Test If No Winner
- Identifying when a RocketForm is in Test Mode
- What’s Coming Next
How A/B Testing Works
You now have the ability to run a basic A/B test of any RocketForms inside of your account. This first version is simple to start. Two variants are displayed at random, with the variant sticking to the user session. The system will track and report conversions for each RocketForm, so you can see which variant drives in more leads.
Details for v1.0
- Default weight of each test is 50/50 — with 50% of the traffic going to the control RocketForm and 50% of the traffic going to the challenger. In the first version, you cannot adjust the weight, however we will give you the ability to do this in the future.
- You can only test two RocketForms at one time. In other-words, A/B testing does not support multi-variant testing at this time.
- When you create a new A/B test the system will automatically create a variant RocketForm from the control. Variant RocketForms do not count against your account limit.
- The system will track total views and conversions. Both of these metrics are unique.
- A/B testing does not currently provide a confidence score and it will not automatically end the test for you. These features are coming in future versions.
Create and Edit an A/B Test
To create an A/B test, follow the steps below:
Go to the RocketForms that you’d like to test and click on the A/B test button inside of the RocketForms Details Page
Step 2 — Start A/B Test
If no tests are in progress then click the “Start A/B Testing” button. If a test is in progress then you’ll need to end your test before you can run a new test for the RocketForms.
Step 3 - Create New Variant
After you click the “Start A/B Testing” button in the step above, a modal pop-up will appear which outlines steps that you need to take to create your A/B test. Click on the Create New Variant button.
After you click on the “Create New Variant” button, you’ll be redirected to the Variant RocketForms.
Step 4 - Edit Variant
The system will automatically create a new variant based on your existing RocketForm after you click on the “Create New Variant” button in the step above. A variant is simply an exact clone of the RocketForm that you want to test and it does not count against your total RocketForm limit.
The variant page is marked with an orange bar that says “You are currently editing a variant RocketForm”
To edit the variant RocketForm, simply make your change somewhere within the variant RocketForms' editor and click Save.
Note: Any changes that you make to the variant will not impact the original RocketForms.
Step 5 - Access the A/B Test Dashboard
To access your test, click on the link in the orange bar: “View this Variant in your A/B test.”
Clicking on the link above will take you straight to the test, which will be in Draft Mode.
You can also access your test at any time from the RocketForm details page by clicking on the “View Results” button.
Start Your A/B Test
After you create, edit, and save your variant RocketForms, your test will automatically be in Draft Mode.
To run your test, simply click on the Green Start Test Button.
Clicking “Start Test” will switch the A/B test from draft to “active” mode.
You can Pause a test at anytime by clicking “Pause Test”
Clicking Pause will switch the A/B test into an in-active state. When a test is paused, the control RocketForm will continue to get all of the traffic until the test is unpaused.
Use the notes feature to help you keep track of what you’re testing.
To add a Note, simply click on the blue pencil icon. This will open up a popup where you can leave your note.
e recommend keeping your notes short and focused on the primary thing that changed with the challenger RocketForm.
Ending a Test
The first version of A/B testing does not calculate a winning variant or display percent increase. This feature is coming in future versions. In the meantime, we recommend using this handy calculator to help identify when to end your test and when you might have a winner: https://cxl.com/ab-test-calculator/
Choose a Winner
Once you have identified a winner, click on the “Choose as Winner” button for the winning variant.
When you choose a winner you’ll have two options:
- End Test and Override Current Default RocketForm: This will end the test, mark the variant as the winner and override the existing RocketForm. *Note — this will completely remove the control from your account.
- End Test and Duplicate as New RocketForm:This will end the test, but instead of overriding the control, it will create an entirely new RocketForm. We recommend using this option if you’d still like to keep the control RocketForm inside of your account for future use.
How to End Test If No Winner Is Determined
You may run a test and not find a clear winner. To end the test if no clear winner is determined, you can either mark the Control as the winner or you can click on the link in the bottom right hand corner to end the test if no clear winner is determined.
To end the test, simply click on the link then click “Yes — End Test” in the prompt.
This will end the test and remove the variant from your account. With the first version of A/B testing, the test will not be stored.
Identifying When a RocketForm is in Test Mode
We added an icon to the dashboard to help you identify when a RocketForm is in test mode. Simply look for the test icon on your Dashboard.
When a RocketForm is in test mode, you’ll also see a banner across the RocketForms Details Page and Inside of the RocketForms Builder.
What’s Coming Next
Here are some future features that we plan to add to A/B testing next:
- Adjust weight beyond 50/50
- Show percent difference
- Recommend winner when statistical significance is reached
- Save past tests
- Email notifications about results