Novelty effect
A temporary change in behaviour because something is new, which fades once visitors get used to it.
Returning visitors may click a new design simply because it is different. Early results then overstate the long-run effect. The opposite, a primacy effect, happens when regulars first resist a change they later accept.
Running whole weeks and watching whether the lift shrinks over time helps tell a lasting effect from a novelty.
A redesigned menu lifts clicks by 20% in the first week and by 4% by the fourth.
Related terms
- A/B testAn experiment that shows two versions of a page to randomly split visitors and compares which one performs better on a chosen goal.
- Sample sizeHow many visitors a test needs per variant to detect the minimum detectable effect with the chosen confidence and power.
- SegmentA subset of a test's visitors, such as mobile visitors or new visitors, whose results can be looked at separately.