In short
Learning goal: by the end of this lesson you can decide, for each traffic source, whether to test the page it lands on, which kind of test to use, and which goal decides it.
Who this is for: Shopify brands doing roughly $1M to $10M a year that spend on paid social, search or email and send that traffic to dedicated pages, collection pages or the homepage.
The promise problem
Every visit starts with a promise. An ad says "bestselling linen sheets, now in sage". An email says "early access to Black Friday prices". A search result says "free returns". The landing page either continues that promise or makes the shopper look for it.
When it doesn't, the shopper does one of three things: hunts through the menu, goes back to the ad platform, or leaves. Google's own description of landing page experience, one of the parts of its Quality Score for search ads, is "how relevant and useful your landing page is to people who click your ad" (Google Ads Help, read 12 October 2026). The idea is not specific to Google: a page should answer the question the visitor arrived with.
That gives landing page tests a clear mechanism to test: does the page continue the promise of the source that sent the visitor?
What to measure at this step
| Metric | Definition | Use it as |
|---|---|---|
| Revenue or profit per visitor | Revenue (or gross profit) from everyone who landed, divided by those visitors | The primary goal |
| Next-page rate | Share of landing visitors who view a second page | A diagnostic |
| Product views per visitor | Product pages viewed per landing visitor | A diagnostic |
| Bounce rate, scroll depth | Share who leave after one page; how far they scroll | Diagnostics, never the decider |
| Page errors, load time | Script errors and slow loads on the landing page | Guardrails |
The primary goal is measured over everyone who landed, not only over those who reached a product page. A landing page that sends fewer but better visitors onward can win. One that sends more visitors onward who then don't buy has only moved the drop one step down.
Why a bounce-rate win isn't one
Bounce rate is the easiest metric to move. A page with less above the fold, a popup, or a "shop now" button with no context will get more visitors to click something. Whether they buy is a separate question, and it is the only one that matters.
Illustrative counts, not data from a real store. Both results computed with Benchmyrk's results engine, 14 days, two versions.
A paid-social landing page, 9,000 visitors per version. The new version moves the product grid above the brand story.
| Goal | Control | New version | Lift | 95% interval | Verdict |
|---|---|---|---|---|---|
| Next-page rate (diagnostic) | 3,780 of 9,000 (42.0%) | 4,005 of 9,000 (44.5%) | +6.0% | +2.5% to +9.6% | Winner |
| Orders per visitor (primary) | 162 of 9,000 (1.8%) | 167 of 9,000 (1.9%) | +3.1% | −16.8% to +27.7% | Inconclusive |
The next-page rate result is clear: more visitors clicked on. The orders result could be anything from a 17% loss to a 28% gain. With one primary goal set before launch, this test is inconclusive, not a winner. The diagnostic tells you what happened on the page; it doesn't tell you whether it made money.
Common problems
- Testing the page, not the match. A redesign of a landing page that serves five ad campaigns tests the average promise. Test one source's page against that source's traffic.
- Mixed traffic in one test. Email subscribers and cold paid traffic respond differently. If both land on the same page, target the test at one source, or look at the split as a description only.
- Too little traffic for the goal. A single campaign's landing page often gets a few hundred to a few thousand visitors a day, and only a small share of them buy. See the example below.
- Split URL redirects. A split URL test sends visitors in the variant to a different URL. The redirect adds a page load and can lose visitors in that version only, which shows up as a sample ratio mismatch. Check the split before reading anything.
Split URL or on-page?
| Situation | Use |
|---|---|
| Two completely different pages (a long-form story page vs a collection) | Split URL test: the variant redirects to its own URL, keeping query parameters |
| The same page with a different headline, hero, order of sections or offer line | On-page test: changes applied to the existing page, no redirect |
| A page built in a page builder you can't edit in the test tool | Split URL test, pointing at the builder's page |
When either would work, prefer on-page. It has no redirect, so the split stays clean and the load time is the same in both versions.
Test ideas, each with its mechanism
- Match the headline to the ad. Mechanism: the visitor confirms in one second that they're in the right place. Target by UTM campaign so each ad set's visitors see their own headline.
- Lead with the product in the ad. Mechanism: an ad for one product that lands on a collection makes the shopper search. Put that product first, with its price and delivery line.
- Repeat the offer exactly. Mechanism: "20% off sitewide" in the email and no mention on the page creates doubt. Show the same words, and make sure checkout gives it.
- Shorten the path for returning customers. Mechanism: email traffic already knows the brand. A page that skips the brand story and goes straight to new arrivals may suit them better than cold traffic.
Illustrative inputs. Computed with Benchmyrk's sample-size formula (two-sided, 95% confidence, 80% power, two versions, even split).
Inputs: 1,500 visitors a day from one paid campaign, 1.8% of whom buy.
| Smallest change in orders per visitor to detect | Visitors per version | Days |
|---|---|---|
| 20% | 23,507 | 32 |
| 10% | 89,839 | 120 |
A single campaign's page can detect only a large change in a month. That's a reason to test bold differences (a different page, not a different button color) on campaign pages, and to save small refinements for pages that get most of your traffic. The sample size and test duration guide explains the inputs.
Exercise (10 minutes, on your own store)
- Open your three biggest traffic sources for the last 28 days (for example: paid social, email, branded search).
- For each, write down the promise: the main line of the ad, email or search result.
- Open the page each one lands on, on a phone. Can you see that promise without scrolling? Write yes or no.
- For the biggest "no", write a one-line hypothesis: "Showing [promise] above the fold for [source] visitors will raise revenue per visitor, because they confirm they're in the right place."
- Check that source's daily landing visitors and conversion rate in the sample-size calculator. Is a 20% change detectable in four weeks?
You don't need Benchmyrk for any of this.
Do it in Benchmyrk
Check yourself
- Your new landing page lowered bounce rate by 15%, and revenue per visitor is inconclusive. What do you report? Inconclusive. Bounce rate is a diagnostic. The primary goal, set before launch, decides.
- When is a split URL test the right choice? When the two versions are completely different pages, or the variant lives somewhere the test tool can't edit. Otherwise prefer an on-page test, which has no redirect.
- Why target a landing page test at one traffic source? Because the mechanism is matching that source's promise. Mixing sources tests an average promise and dilutes the effect.