The short answer
- If you already run promotions, start with the discount. You're giving it away anyway, the question comes up every month, and a test is easy to reverse. The usual finding worth having is that a smaller or differently shaped offer earns more per visitor.
- If you rarely discount and suspect your price is off, start with the price. A price is permanent and touches every order, so getting it right is worth more, but the test needs more care and usually more visitors.
- Either way, decide on profit per visitor. Orders and conversion rate will usually favour the cheaper option.
Two different questions
| Discount test | Price test | |
|---|---|---|
| Question | How much to give away, in what form, and for which baskets | What shoppers will pay for this product |
| What changes | An offer at checkout: percent off, amount off, spend tiers, buy X get Y | The price on the product page, the collection page and at checkout |
| Who sees it | Every visitor in the group, often store-wide | Visitors to the tested products |
| How long the answer lasts | Until the next promotion | Until your costs or competitors change |
| Reversing it | End the offer | Change the price back, after shoppers have seen it |
| Risk | Training shoppers to wait for a code | Shoppers noticing different prices; ads showing a different price |
The break-even maths
Before you test either, work out what the test has to show. Pricing textbooks call this the break-even sales change: how many more (or how many fewer) units keep gross profit where it is. It depends only on the size of the change and your contribution margin.
discount d: extra units needed = d / (m − d) price increase p: units you can lose = p / (m + p)- Profit per unit today
- $24.00
- 20% off: price and profit per unit
- $32.00, $16.00
- Units needed to keep the same profit
- +50%
- 10% higher: price and profit per unit
- $44.00, $28.00
- Units you can lose and still earn the same
- −14.3%
Illustrative numbers. Product cost only; payment fees and shipping lower the margin a little further.
| Discount | 40% margin | 50% margin | 60% margin | 70% margin |
|---|---|---|---|---|
| 10% | +33% | +25% | +20% | +17% |
| 20% | +100% | +67% | +50% | +40% |
| 30% | +300% | +150% | +100% | +75% |
| Increase | 40% margin | 50% margin | 60% margin | 70% margin |
|---|---|---|---|---|
| 5% | −11% | −9% | −8% | −7% |
| 10% | −20% | −17% | −14% | −13% |
| 15% | −27% | −23% | −20% | −18% |
The asymmetry is the point. A 20% discount at a 50% margin needs two-thirds more units just to stand still; a 10% price increase at the same margin can lose a sixth of its units and still earn as much. That doesn't mean prices should go up. It means a discount has to work much harder than it looks, and a price increase has more room than it feels like it has. Only a test on your store says which way your shoppers go: how strongly they react is the price elasticity you're measuring.
What each is like to run on Shopify
- Discounts are native. Shopify has percent, amount, buy X get Y and free shipping discounts, automatic or by code. To test them, each group of shoppers needs its own offer at checkout and a banner that says exactly what checkout gives.
- Discounts don't combine unless you set them to. On Shopify each discount lists which classes (product, order, shipping) it combines with, shoppers can use at most five product or order codes and one shipping code on an order, and a store can have at most 25 automatic discounts active. Decide how a test offer meets your codes, loyalty perks and email offers before it starts.
- Prices need the page and checkout to agree. The tested price has to show on product and collection pages and be charged at checkout, for the same shopper, every time. Benchmyrk does it with a Shopify discount at checkout, which works on every plan; that's why its tested prices are at or below the store price (see price testing on Shopify without Plus for the other ways and their limits).
- Prices reach places discounts don't. Your Google Shopping feed, marketplaces and price comparison sites carry the store price. Google Merchant Center asks for the price in your product data to match the landing page and checkout, so test prices away from products you advertise heavily, or keep the test short.
- Price tests need more visitors per product. A price test only reaches shoppers who view the tested products, while a store-wide offer reaches everyone. Check the sample size for the traffic those products actually get.
Fairness and the rules
Shoppers accept that discounts vary: different codes in different emails is normal. Different prices for the same product feel different, so keep price tests short, the differences modest, and be ready to honour the lower price if a shopper asks.
If you sell in the EU, two rules from Directive (EU) 2019/2161 are worth knowing. An announced price reduction must show the lowest price you applied in at least the previous 30 days. And traders must tell consumers when a price has been personalised on the basis of automated decision-making. Whether a randomised price test counts as personalised pricing is a legal question; ask whoever advises you on consumer law before testing prices for EU shoppers.
A sequence that works for most stores
- Format at equal cost. 15% off against $15 off on a $100 average basket, or a flat discount against spend tiers that cost about the same. Cheap to run, and the answer applies to every promotion after it.
- Depth. Your usual discount against a smaller one. Work out the break-even first; if the smaller offer earns the same profit per visitor, keep it and bank the margin.
- Threshold. Offers and free shipping from a minimum order grow baskets as well as orders; the free shipping threshold calculator gives the break-even.
- Price. Once you know how shoppers respond to offers, test the price of a few high-traffic products, in a normal month, away from sales.
Planning around a big sale? The Black Friday guide covers which of these to run before the peak and which to leave until January.