Shipping threshold A/B tests for agency clients

A shipping threshold A/B test gives random groups of a store's shoppers different free-shipping minimums, or the current shipping offer, at the same time. Each group sees its own threshold on the page and gets it at checkout, and the groups are compared on revenue or gross profit per visitor.

The glossary entry for a shipping test has the definition and what to judge one on.

Plan whole weeks, at least two, so the test ends by November 15 and the winner is live before the sale. For 2026 that means launching by Sunday, November 1 at the latest; a three-week test from October 25 is safer.

Threshold tests change order value as well as conversion, so they need more visitors than a conversion-only test. Keep the client's big promotions out of the test window.

Ask each client for:

• Average order value over the last 60 to 90 days, excluding wholesale.
• Gross margin on products (after the cost of goods).
• What they charge shoppers for shipping today, per order.
• What shipping costs them per order: carrier fees, packaging and any fulfilment fee charged per parcel.
• The share of orders already above the threshold you want to test.

Clients often know what they charge and not what it costs. Fulfilment invoices from the last two months are the quickest source.

Free shipping lowers what each order earns by the shipping charge given up. To stay level, the client needs enough extra orders, or bigger baskets, to make that up.

This is the simplest case: free shipping on every order, with baskets unchanged. A threshold changes both how many orders qualify and how big baskets get, so run each client's numbers through the free-shipping threshold calculator, which handles the share of orders above the threshold. The free-shipping threshold guide walks through the full break-even.

For most clients, test one threshold a little above their average order against their current shipping offer. Set it high enough that some carts have a reason to grow, and low enough that a real share of shoppers can reach it.

Useful designs:

• Two thresholds against the current offer, for a client with high traffic: for example free from $50 and from $75.
• Half-price shipping on every order against free shipping from a threshold, to compare lowering the barrier for everyone with rewarding bigger carts.
• Free standard shipping only, with express kept at full price, for clients whose margins can't absorb express.

Each arm can show a progress bar toward its own threshold. Make sure every banner and bar promises exactly what checkout will give that arm.

Shopify allows one shipping discount per order: multiple shipping discounts can't apply to the same order (Shopify Help Center, read October 11, 2026). If the client already runs its own free-shipping discount, a shopper who qualifies for both gets the better one, and the test measures the difference only where the client's own offer doesn't apply.

Before launch:

• List the client's active shipping discounts and any free-shipping codes in planned emails.
• Decide whether to pause the client's own threshold during the test, or test only above or below it.
• Check how each minimum is counted. Benchmyrk's shipping tests count the cart minimum before discounts (the line subtotal), unlike a Shopify free-shipping discount, which counts discounted prices. An $80 cart with 10% off is $72 after the discount: it reaches a $75 test minimum, but not a $75 Shopify free-shipping minimum.
• Count automatic discounts. Shopify allows up to 25 active automatic discounts per store, app discounts included (Shopify Help Center, read October 11, 2026). Benchmyrk's price, offer and shipping tests use up to four of them.
• Agree what the sale-week shipping offer will be, and that it's built from the test's result.

Clients selling in several currencies need extra care. In Benchmyrk, fixed amounts and minimums are set in the store currency and converted for shoppers paying in another currency, the way Shopify converts its own discounts. Percent offers are the same in every currency. A "$75" threshold therefore becomes a converted amount in other markets, which may not be a round number shoppers recognize. For clients where international orders matter, consider testing only the home market, or using a percent-off-shipping offer. The help article on currencies and subscriptions covers what applies where.

Judge on revenue per visitor or gross profit per visitor, with the 95% interval, not on conversion rate or order value alone. A threshold can raise order value and lower conversion; only per-visitor revenue or profit tells you which effect won. If the result is inconclusive by November 15, the honest answer to the client is "keep the current offer for the sale and re-test in January", not "the bigger threshold looked better".

• Using what the client charges as the cost. Break-even depends on what shipping costs them.
• One threshold for every client. Order values and margins differ, so the right threshold differs.
• Judging on conversion rate. A threshold works through basket size; read revenue or profit per visitor.
• Overlapping shipping discounts. Only one applies per order; plan around the client's own.
• Thresholds that become odd amounts in other currencies. Test the home market or use a percent offer.

• [ ] Average order, margin, shipping charged and shipping cost collected
• [ ] Break-even worked out with the calculator
• [ ] One test per store; thresholds set just above the average order
• [ ] Existing shipping discounts listed and handled
• [ ] Currencies and markets checked
• [ ] Banners and progress bars match what checkout gives
• [ ] Primary goal: revenue or gross profit per visitor; ends by November 15

Setting a threshold for one brand instead? The brand track covers free-shipping thresholds for BFCM baskets. The general timeline is in the guide to testing before Black Friday.