Guardrail metric
A metric a test must not make worse, watched alongside the goal the test is trying to improve.
Guardrails protect what the test isn't about. A variant can raise add-to-cart rate while hurting orders, or raise orders while raising refunds. Guardrails catch that.
They are set before launch, often with an action when they fire: an alert, or pausing the test.
A test of a bolder discount banner uses orders as its goal and refund rate and revenue per visitor as guardrails.
Related terms
- Primary goalThe one metric a test is judged on, chosen before it starts.
- Revenue per visitor (RPV)Total order revenue divided by visitors in a variant: a goal that combines how many visitors buy and how much they spend.
- A/B testAn experiment that shows two versions of a page to randomly split visitors and compares which one performs better on a chosen goal.