Benchmyrk

Guide

Price testing on Shopify without Plus

A price test only means something if each shopper pays the price their page showed. On Shopify that rules out most shortcuts. Here are the options, what each needs, and how Benchmyrk runs price tests on every plan.

Benchmyrk teamPublished 5 min read

Why price tests are hard on Shopify

A Shopify product variant has one price (per market). You can't change it per visitor, and changing it for everyone is not a test: it is a before-and-after comparison, and anything else that changed that week (a campaign, a holiday, the weather) is mixed into the result.

A real price test splits shoppers at random and keeps both groups running at the same time. That needs two things: the storefront must show each group its own price, and checkout must charge it. The second part is where Shopify's plans matter.

The options, compared

Ways to charge a different price per shopper group on Shopify
MechanismShopify plansCan it raise a price?What the shopper sees at checkout
Discount Function (an app's automatic discount)Every plan, in a public appNo, discounts only lowerThe store price struck through, the lower price and the discount's message
Cart Transform with update operationsShopify Plus and development stores onlyYesThe new price as the line's own price
Duplicate products or variants per groupEvery planYesA different product, with its own inventory, reviews and feed entries
Change the price for everyone, compare weeksEvery planYesOne price; not a randomized test
Plan limits from Shopify's developer docs, checked on 9 October 2026 (sources below).

Shopify says it plainly: "Only development stores or stores on a Shopify Plus plan can use apps with update operations" in the Cart Transform API. Functions in general are open to every plan when they ship in a public app; only custom apps that contain functions need Plus. So on a non-Plus store, the clean way to charge a tested price is a discount that lowers the store price for some shoppers.

How Benchmyrk runs price tests

  1. Set the arms at or below the store price. A discount can only lower a price, so the Shopify price is the highest price in the test. To test a higher price, raise the price in Shopify to the highest arm, read the prices again in Benchmyrk, and set the other arms, the control included, lower. Visitors outside the test see and pay the Shopify price.
  2. The storefront shows each arm's price. A small script (bc-price.js, loaded only while a price test runs) replaces the printed price of the tested products on product pages, collection cards and other price elements with the arm's price.
  3. Checkout charges it. The shopper's arm travels with the cart as a cart attribute. Benchmyrk's discount, built on Shopify's Discount Function API, lowers each tested line to the arm's price.
  4. Results compare money, not just orders. Revenue and gross profit per visitor, plus units sold and the average price paid per arm, with the same statistics as every other test.
A 10% lower price on one productExample
Shopify price (control)
$128.00
Arm B price
$115.20
Product page, arm B
$115.20
Checkout, arm B
$128.00 struck through, $115.20, "Store price"
Visitors outside the test
$128.00 everywhere

Illustrative prices. "Store price" is the default checkout message; you can change it.

Never charged more than the page showed

The rule Benchmyrk holds price tests to: a shopper is never charged more than the price their page showed, and a tested price is shown only where it is charged.

  • When a test starts, checkout starts charging the test prices before the storefront shows them. When it stops, the storefront stops showing them before checkout stops charging. During the switch a shopper can pay less than the page said, never more.
  • Pausing or ending a test stops the checkout discount at once; storefront pages follow within about a minute.
  • If someone turns the discount off in Shopify, Benchmyrk notices within about ten minutes, takes the price test off the storefront and notifies the store's owners and admins.
  • The discount only applies to the Shopify prices the test was set up with. If a price changes in Shopify during the test, shoppers see and pay the new Shopify price for that product.

What to measure

A lower price usually converts better, and a higher one usually sells fewer units. Conversion rate alone will nearly always favour the cheapest arm. Judge a price test on revenue per visitor, or better, gross profit per visitor, which takes off product costs, payment fees and shipping.

Profit needs product costs in Shopify. Benchmyrk gives no profit verdict while costs are known for less than 90% of order value. Plan for revenue goals to need more visitors than conversion goals: see Sample size and test duration.

Good to know

  • One currency: shoppers browsing in another currency (Shopify Markets) are left out of the test.
  • Products with selling plans (subscriptions) can't be in a price test.
  • Savings badges your theme computes ("Save $22") and instalment widgets keep using the Shopify price.
  • On a tested product's page, Buy it now and accelerated checkout buttons are hidden for shoppers in the test, in every arm alike, because those checkouts skip the cart attribute.
  • Themes that add to cart through custom code that skips the cart attribute charge the normal price; the cart then shows that price before payment. Start with a small share of traffic and check the storefront.
  • Up to 10 products and 100 variants per test, and one live price test per product.
  • Variants of one product that share a Shopify price need the same test price.
  • The discount needs two optional permissions (read and write discounts) that an owner or admin grants from the Prices tab, not at install.

Is it fair to show different prices?

Price tests are common in ecommerce, and the safeguard that matters most is consistency: each shopper sees one price and pays that price. Keep tests short, keep the differences modest, and be ready to honour the lower price if a shopper asks about it. Rules on price differences vary by country, so check with your adviser if you sell somewhere with specific pricing rules.

Before you start

  1. Decide the question: which product, which prices, and whether you're judging on revenue or profit.
  2. Add product costs in Shopify if you want profit per visitor.
  3. Set the Shopify price to the highest arm and read prices again.
  4. Check the sample size for the traffic those products get.
  5. Preview each arm on the product page, a collection page and the cart.
  6. Start with a small traffic share, place a test order in each arm, then widen it.

Sources

  1. Shopify, Cart Transform Function API ("Only development stores or stores on a Shopify Plus plan can use apps with update operations")
  2. Shopify, About Shopify Functions (availability by plan)
  3. Shopify, Discount Function API

Put it to work on your store.

We'll walk through results and setup on demo data and answer your questions. Or start the 14-day free trial on Shopify.