Shopify Rollouts vs A/B testing apps: what Shopify's own tools can test

A rollout is a scheduled set of changes to your store, created under Markets › Rollouts in the Shopify admin. Shopify describes three types:

• Launch: merges your changes into the store permanently, now or at a set time.
• Event: publishes changes for a set period, then rolls them back. Useful for a sale weekend.
• Experiment: shows the changes (the treatment) to part of your eligible visitors and compares them with the rest (the control). A new Experiment starts at a 50/50 split, which you can change.

Rollouts are available on the Basic plan or higher; Shopify says "Experiments are available to stores on the Grow plan or higher." Store owners and admins can manage them, and other staff need the Rollouts permission plus permission for each type of change.

• Your main theme. Edit it in the theme editor inside the rollout, or replace it with another theme. Edits made in a rollout apply only to that rollout, so the live store stays as it is.
• Checkout and accounts. Replace your published checkout and accounts configuration for the treatment group.
• Catalogs. Set market catalogs you created to Active or Draft. An active catalog publishes its prices and product availability, so a catalog change is the closest Rollouts gets to a price change.
• Discounts. Activate or deactivate existing discounts for the visitors in the rollout. Shopify's developer changelog announced discounts in Rollouts on 1 October 2026.

And the limits Shopify lists:

• One theme change, one checkout change and one discount change per rollout; a rollout can include more than one catalog change.
• No Liquid template changes, and no vintage themes.
• Experiments, and any rollout shown to less than 100% of visitors, apply to the online store only. Headless and custom storefront checkouts can't be tested.
• A discount, or a catalog, can be in only one unfinished rollout at a time. Catalogs assigned directly to a B2B company location can't be added.
• If the treatment or the traffic share is below 100%, the rollout needs an end date. The default end date is 90 days after the start.
• Editing the treatment or control while an Experiment runs can affect its results; Shopify warns about discount edits that could.

• Shopify says plainly: "You can't customize the available metrics for your rollout."
• For discounts: "Discount-specific metrics, such as the revenue or the orders attributed to a discount, aren't available."
• The metrics cover visitors on your online store, cart and checkout, not headless storefronts or POS.
• The Rollouts help pages don't describe a significance test, confidence intervals or how to tell when a result is ready. Shopify's Smart Pricing experiments do: their results are marked Conclusive at 95% confidence.

Why this matters: a change can raise conversion rate and still lose money, for example a discount that converts more shoppers at a lower margin. For anything that touches price, offers or shipping, judge the test on revenue per visitor or gross profit per visitor, and decide the sample size before you start. See how to read an A/B test result.

Two routes exist, both with conditions.

• Smart Pricing A/B experiments. Shopify's Smart Pricing app runs price experiments for select stores in early access. Shopify lists the requirements: Grow plan or higher, based in the United States and selling to the US, the new Shopify Markets with one catalog for the US market, and at least 10 products each with more than 25 sales a month. One experiment at a time, up to 250 products, one test price per product, and shoppers split 50/50. It tests prices on the online store only, and isn't compatible with third-party personalization apps.
• A catalog change in a Rollouts experiment. Setting a market catalog to Active publishes its prices to the treatment group. Shopify warns that the online store can then show different prices from Google and other channels, and the experiment metrics listed above don't include revenue or profit per visitor.

For stores outside the US, stores on the Basic plan, or tests judged on profit, a price-testing app is still the practical option. Our guide to Shopify price testing without Plus explains the mechanisms apps use and their limits.

Because an Experiment can activate an existing discount for its treatment group, you can now compare, say, a free-shipping discount against no discount without an app. Things to know before you do:

• The discount must already exist; the rollout only turns it on or off for its visitors. Shopify's discount combination settings still apply, so the control and treatment groups can end up eligible for different discounts.
• At partial traffic, the discount applies on the online store only, according to Shopify's developer changelog.
• Revenue and orders attributed to the discount aren't reported, so you'll need another way to see whether the offer paid for itself.
• Apps that read your discounts through an Admin API version older than 2026-10 may not see a discount while a rollout is changing it. If another app manages or reports on your discounts, check that it has been updated.

For the maths behind a shipping offer, see our free shipping threshold test guide.

• You set the share of eligible visitors and the split between treatment and control. Eligibility follows the markets of the changes you added.
• Shopify's rollout pages don't describe targeting by device, traffic source, customer segment or behaviour.
• A visitor is assigned to only one rollout at a time for a given resource. Launch rollouts take their traffic first; the rest is shared among active Experiments.
• External channels "don't receive rollout information", so Google Shopping and other feeds keep showing what's published.

• You're on Grow or higher and want to try a new theme, a theme redesign or a checkout configuration on part of your traffic before it goes to everyone.
• Conversion rate, add-to-cart and reached-checkout rates answer your question.
• Two versions are enough, and you don't need to target a device, a campaign or a customer segment.
• You want to schedule a launch or a sale and have Shopify roll it back automatically when it ends.

• Don't test the same thing twice at once. A Rollouts theme experiment and an app's test on the same pages put shoppers in two tests, and neither result is clean.
• Use Rollouts for scheduling launches and sales, and the app for the tests you judge on money.
• If you put a discount into a rollout, check how your testing or discount apps see it before you start.