A/B test
An experiment that shows two versions of a page to randomly split visitors and compares which one performs better on a chosen goal.
In an A/B test, each visitor is assigned at random to the current version (the control, A) or a changed version (the variant, B), and keeps that assignment on later visits. Because the split is random and both versions run at the same time, a difference in results can be attributed to the change rather than to the season, a campaign or the mix of visitors.
The result is judged on a goal set before the test starts, such as orders or revenue per visitor, with statistics that say how sure the difference is.
Half of a store's product-page visitors see the current "Add to cart" button, half see one with a delivery date under it. After three weeks, the version with the delivery date has a higher order rate and a 95% interval that stays above zero.
Related terms
- ControlThe unchanged version in a test, which every variant is compared against.
- VariantA changed version of a page or experience in a test, compared against the control.
- A/B/n testAn A/B test with more than one challenger: the control is compared with two or more variants in the same experiment.
- Split URL testA test that sends part of the traffic to a different URL, such as a redesigned landing page, and compares the outcomes.