The short answer
- You have about five weeks. From Tuesday 13 October to the change freeze we suggest on Monday 16 November is roughly 34 days of testing time. Black Friday is Friday 27 November 2026 and Cyber Monday is 30 November.
- At your traffic, that is one round of tests, not two. A store with 60,000 visitors a week needs about 15 days to detect a 10% change in conversion rate on a site-wide test (Example below). Tests that only see product-page traffic, or that are judged on profit, take three to four weeks.
- So pick 2 to 4 tests whose answers you will use in the sale: the offer format, the free-shipping threshold, the landing page your ads and emails point to, and the product page of your best sellers.
- Judge each one on profit per visitor, decided before launch. A deeper discount usually sells more orders. Whether it earns more is a different question.
- Write down who owns what before the first test starts: your team, your agency and every app vendor who can change the storefront.
Who this is for: Shopify brands doing roughly $1M to $10M a year, with enough traffic to run a few tests at once and a team (often with an agency) that ships changes every week.
The 2026 timeline
| When | What happens |
|---|---|
| Tue 13 Oct to Sun 18 Oct | Choose the tests, write each hypothesis and primary goal, check tracking (part 2 of this series). |
| Mon 19 Oct | Latest sensible start for tests that need three to four weeks. |
| Sun 25 Oct | Latest start for a three-week test that should end before the freeze. |
| Sun 1 Nov | Latest start for a two-week test. |
| 2 Nov to 15 Nov | Read results, ship winners, build and QA the sale offer and banners. |
| Mon 16 Nov | Freeze. No theme publishes, new apps, or new tests that touch prices, discounts, shipping or checkout. |
| Thu 26 Nov to Mon 30 Nov | Run what you decided. Watch guardrails and orders daily. |
| December | Wait for refunds, then analyze (part 10). |
The freeze date is our suggestion, not a Shopify rule. Part 3 covers how to move it if your team relies on an agency or app vendors for changes.
Step 1: Work out what your traffic can detect
Every test plan starts with four inputs: the baseline rate of your primary goal, the smallest change worth detecting (the minimum detectable effect, MDE, as a relative change), the confidence level (95%) and the power (80%). Those four give the number of visitors each version needs. Divide by the visitors per day who will actually reach the tested page and you have the test length. The sample size and test duration guide explains each input, and the sample-size calculator does the arithmetic.
Two things catch brands out:
- Only visitors who reach the tested page count. A product-page test on your top 20 products sees the share of traffic that lands on those pages, not your whole store.
- Revenue and profit goals need more visitors than conversion rate. Order values vary a lot from one order to the next, and that spread is noise the test has to see through.
Illustrative inputs, not data from a real store. Computed with the same sample-size formula Benchmyrk's test planner uses (two-sided, 95% confidence, 80% power, an even split).
Inputs: 60,000 visitors a week (about 8,571 a day), a 2.5% conversion rate, and a 10% relative MDE (2.5% to 2.75%).
| Test | Visitors per version | Days at this traffic |
|---|---|---|
| Two versions, every visitor enters | 64,199 | 15 |
| Two versions, product pages only (60% of visitors) | 64,199 | 25 |
| Three versions, every visitor enters | 77,746 | 28 |
| Two versions, judged on revenue per visitor (order value $80, spread $60) | 96,542 | 23 |
| Two versions, judged on profit per visitor (gross profit $32 an order, spread $26) | 102,674 | 24 |
Turned around: with every visitor in a two-version test, 21 days can detect a change of about 8.5% in conversion rate, and 14 days about 10.4%.
What this means for the plan:
- A test started on Monday 19 October that needs 24 days ends on Wednesday 11 November. That leaves time to read the result and ship the winner before the freeze. A 28-day test started the same day ends on Sunday 15 November, which leaves no time at all.
- A third version costs almost two weeks here. Before the freeze, two versions is the default.
- If your conversion rate or traffic is lower, every number above grows. Run your own figures through the calculator before you commit.
Step 2: Pick 2 to 4 tests you will actually use
The question for every candidate: will the answer change what you do on 27 November? If not, it can wait until January.
- Offer format. Percent off against an amount off at the same cost, a flat discount against spend tiers, or buy X get Y. Settle the format in October so the sale runs the version that earns most per visitor. Part 4 has the break-even maths.
- Free-shipping threshold. Sale-week baskets are different, so test the threshold on normal October traffic and decide whether to keep it for the peak. Part 6 covers it.
- The landing page your ads and emails will send people to. A gift guide, a BFCM collection page or a best-seller page. Layout, product order and how the offer is explained.
- The product page of your best sellers. Delivery cut-off messages, returns information near the add to cart button, and how the discount is shown.
Leave price tests for full-price products that are not in the sale, and end them before the freeze (part 5). Leave checkout changes to Plus stores with time to spare (part 7).
Running tests at the same time. Tests on different pages can usually overlap: a shopper who sees a new collection layout and a new product page is in two tests, and with even splits each test's comparison stays fair. Tests that change the same element, the same prices or the same discount should be mutually exclusive, so no shopper is in both. That splits traffic between them, so each takes longer. Plan for it.
Step 3: Back-plan each test from the freeze
For each test, write one line before you build anything:
- Hypothesis: what you change, for whom, and why you expect it to matter.
- Primary goal: one metric that decides the test. For offers, thresholds and anything that changes what shoppers pay, use profit per visitor. For layout tests where cost doesn't change, revenue per visitor is fine.
- Guardrails: metrics that must not get worse, such as checkout completion, page errors or missing page elements.
- Planned length: from the calculator, at the traffic that page actually gets.
- Start and end dates, with the end at least two working days before 16 November.
- The decision: what you will do if it wins, loses or is inconclusive. "Inconclusive" is a common result at these traffic levels. Decide in advance that it means "keep the current version for the sale".
A discount can lift conversion rate and still lower revenue per visitor, and lower profit per visitor even more; the revenue per visitor vs conversion rate article explains why.
Step 4: Decide who owns what
At this size, the people who can change your storefront usually include an in-house marketer, a developer or agency, and several app vendors. Write a one-page owner list:
| Area | Owner | Can change after 16 Nov? |
|---|---|---|
| Theme code and theme publishes | Developer or agency | No |
| Apps and app embeds | Ecommerce lead | No new installs |
| Discounts and codes | Marketing | Only the planned sale discounts |
| Tests | Growth lead | Only scheduled ends and guardrail stops |
| Email and ad landing pages | Marketing | Copy only, reviewed |
Two Shopify changes are worth checking on that list this year:
- Shopify Scripts no longer run. Shopify ended Scripts on 30 June 2026. If last year's BFCM promotions used Scripts on Plus, that logic now has to come from Shopify Functions, through a discount app or a custom app.
- Automatic discounts have a limit. Shopify allows up to 25 active automatic discounts per store, app discounts included (Shopify Help Center, read 11 October 2026). Benchmyrk's price, offer and shipping tests use up to four of them. Many discount and bundle apps use them too. Count them before the freeze.
Mistakes to avoid
- Starting with a test you won't use in the sale. A homepage hero test is fine in January. In October it takes traffic from tests whose answers matter in four weeks.
- Planning with sitewide traffic for a page-level test. Use the visitors who reach the tested page.
- Adding a third or fourth version "while you're at it". At these traffic levels each extra version adds weeks.
- Changing the primary goal after launch. Choosing the metric that looks best after the fact is how false winners happen.
- Calling a winner because it's ahead on day 3. See why repeatedly checking a fixed test inflates false winners. Benchmyrk uses always-valid sequential p-values, so you can look at any time, but it gives no verdict before 7 full days, because weekdays and weekends differ.
Checklist
- Weekly visitors to each page you want to test, and its conversion rate.
- Planned length for each test from the calculator.
- Two to four tests, each with a hypothesis, one primary goal and guardrails.
- Start and end dates that finish at least two working days before 16 November.
- Overlapping tests on the same element set as mutually exclusive.
- An owner list for the storefront, with what each owner may change after the freeze.
- Tracking checked (part 2) before the first test starts.
Running this for several clients? The agency track on the BFCM hub covers planning across a client portfolio. The full season plan is in What to test before Black Friday and Cyber Monday on Shopify.