Chance to beat control
The probability, given the data so far, that a variant's true rate is higher than the control's.
Also called probability to beat control
Chance to beat control is a Bayesian probability. At 50% the data can't tell the two apart; near 100% the variant is very likely better; near 0% the control is.
It says how sure, not how much: a variant can have a 99% chance to beat control with a lift too small to matter, so read it next to the lift and its interval.
A checkout banner shows a 96% chance to beat control with a lift of +4% (interval −1% to +9%): probably better, but possibly by very little.
Related terms
- Bayesian A/B testingAnalysing a test by updating a probability model with the data, so results read as probabilities such as the chance a variant beats the control.
- LiftHow much better or worse a variant performs than the control, as a percentage of the control's value.
- Confidence intervalA range of values for the true effect that is consistent with the data, at a stated confidence level such as 95%.
- P-valueThe probability of seeing a difference at least as large as the observed one if the variant actually made no difference.
Where it comes up
- How to read an A/B test result · Guide