Minimum detectable effect (MDE)
The smallest lift a test is planned to detect reliably; it sets how many visitors the test needs.
Also called MDE
The minimum detectable effect is the smallest true difference worth finding, usually stated relative to the baseline ("10%"). Together with the baseline rate, the confidence level and the power, it sets the sample size.
Smaller MDEs need far more visitors: halving the MDE roughly quadruples the sample. Choose it from what is worth acting on, not from what you hope to see.
At a 3% baseline, a 10% MDE means planning to detect 3.3%. That takes about 53,000 visitors per variant at 95% confidence and 80% power.
Related terms
- Sample sizeHow many visitors a test needs per variant to detect the minimum detectable effect with the chosen confidence and power.
- Statistical powerThe chance that a test detects an effect of a given size when that effect is really there.
- LiftHow much better or worse a variant performs than the control, as a percentage of the control's value.