In short
Learning goal: by the end of this lesson you can choose cart tests that fit each client's economics, set up profit per visitor with the client's cost data, and plan for the extra traffic a revenue or profit goal needs.
Who this is for: agency strategists asked by several clients to "raise average order value".
"Raise AOV" is a frequent brief for agencies, and one of the easiest to deliver badly. Average order value can rise while the store makes less money: fewer people buy, or the add-on that raised the basket is sold at a discount that eats the margin. The goal is more profit per visitor, and AOV is one way to get there.
What to measure at this step
| Metric | Definition | Primary goal or diagnostic |
|---|---|---|
| Profit per visitor | Gross profit from visitors in the arm ÷ those visitors | Primary, when product costs are known |
| Revenue per visitor (RPV) | Revenue ÷ visitors (explainer) | Primary when costs aren't complete |
| Average order value (AOV) | Revenue ÷ orders | Secondary: explains the result |
| Checkout-start rate | Share of cart visitors who start checkout | Diagnostic |
| Attach rate | Share of orders that include the add-on or bundle | Diagnostic |
| Threshold reach | Share of carts at or above a free-shipping threshold | Diagnostic |
| Checkout completion | Share of checkouts that become orders | Guardrail |
Step 1: Fit the test to the client's economics
The same cart test is a different bet for each client. Before choosing one, collect:
- AOV and its spread. How much order values vary. A store where most orders are one $70 item behaves differently from one where orders range from $20 to $500.
- Gross margin by product. The add-on with the best attach rate is useless if its margin is thin.
- Shipping cost per order, for threshold tests. See part 6 of the agency BFCM series for the break-even.
- Existing discounts, which may combine with anything you test.
Then match the test to the client:
| Client economics | Good first cart test |
|---|---|
| Low AOV, decent margin, shipping eats profit | A free-shipping threshold just above the average order |
| One hero product, accessories with high margin | A cart add-on of one accessory |
| Products often bought together | A 2 to 3 product bundle, at full price first |
| High AOV, considered purchase | Leave the cart alone; test reassurance in checkout instead |
Step 2: Get product costs from the client
Profit per visitor is only as good as the cost data behind it. Shopify itself says profit is reported "only for products and variants that had cost recorded at the time they were sold" (Shopify Help Center, Profit reports, read October 12, 2026). Ask each client to:
- Fill in the cost per item for every product in the test, before launch.
- Confirm their payment processing fees.
- Give you a shipping cost per order, or per order band.
Costs added after an order don't apply to that order. Get them in before the test starts.
Step 3: Plan for revenue goals needing more traffic
Revenue and profit vary much more from visitor to visitor than a yes-or-no conversion, because order values vary. That makes revenue goals slower to read.
(an illustrative client, not a real store)
The client's cart test sees 5,000 visitors a day, split evenly between two arms. 3% of them buy, the average order is $70 and order values have a standard deviation of $50. At 95% confidence and 80% power, to detect a 10% change:
| Primary goal | Visitors per arm | Days |
|---|---|---|
| Conversion rate | 53,211 | 22 days |
| Revenue per visitor | 77,453 | 31 days |
Judging the test on revenue per visitor takes about 9 more days than judging it on conversion. It's still the right goal, because a cart test that raises order value can lower conversion, and only revenue or profit per visitor shows which effect won. Plan the extra days in the client's calendar from the start.
The revenue sample size uses the same formula as the conversion one, with the variance of revenue per visitor in place of the variance of a conversion. The sample size calculator handles conversion goals; for revenue, use the client's order-value spread and allow for more time.
Step 4: Decide on large orders before launch
One wholesale-sized order can decide a revenue test on its own. Decide before launch whether to cap large orders, at the same level for every arm, and write it in the test plan. Deciding after you've seen which arm got the big order is choosing the result.
Step 5: Read it honestly
Judge on profit per visitor if the client's costs cover nearly all the revenue in the test, otherwise on revenue per visitor. Look at AOV, attach rate and checkout starts only to explain the result. Refunds reduce credited revenue, so don't call a cart test the day it ends if the client's refund window is long.
Common problems
- AOV as the primary goal. It can rise while revenue per visitor falls.
- Discounted bundles judged on revenue. The discount is a cost; profit per visitor shows it.
- Missing product costs. No complete costs, no profit verdict.
- Two threshold tests at once. Only one shipping discount applies per order; run one at a time.
- Deciding the cap after the fact. Choose it before launch.
Test ideas, with the mechanism behind each
| Idea | Mechanism | Judge on |
|---|---|---|
| Free shipping from just above AOV | Gives a reason to add one more item | Profit per visitor |
| One accessory offered in the cart | Small, relevant addition at the moment of buying | Profit per visitor; attach rate as a diagnostic |
| A bundle at full price | Convenience without a margin cost | RPV; AOV as a diagnostic |
| A progress bar toward the threshold | Makes the threshold visible | Profit per visitor; threshold reach as a diagnostic |
A 10-minute exercise on your own client
- Pull the client's last 60 days of orders: AOV, the spread of order values, and the share above $75 or whatever threshold you'd test.
- Check how many products in the cart's top 20 have a cost per item filled in.
- Pick the row in the Step 1 table that fits the client.
- Write the test, the primary goal and the large-order rule.
Do it in Benchmyrk
Check your understanding
- AOV rose and revenue per visitor fell. What happened? Fewer visitors bought. The bigger baskets didn't make up for the lost orders. The test lost on its primary goal.
- Only 60% of the revenue in the test has product costs. What do you judge it on? Revenue per visitor. Profit per visitor needs costs on nearly all the revenue.
- The largest order in the test was many times the average and landed in the variant. Can you cap it now? No. A cap has to be chosen before launch and applied to every arm. Report the result with and without it, and say which was planned.