Experiment report · 1 June–9 August 2026 · 70 days

Choose on purchases, not clicks

Calculating experiment results…

Recommended next step—Subject to the measurement checks below

F versus current page A · purchases per visitor——

Experiment coverage—Eight randomized versions; visitor-day records may include repeat visitors

Product decision. Calculating…

The outcome that matters: completed purchases

Rates use each version’s total purchases divided by its total recorded visitors—not an average of daily percentages. The chart shows the estimated rate and an approximate 95% interval for each version.

Version	Visitors	Purchases	Purchase rate	Versus A	95% interval for difference	Adjusted p

Differences are in percentage points (pp). Intervals and p-values are illustrative, visitor-level binomial calculations; see the inference limitation below. Adjusted p-values use Holm’s correction for the seven comparisons with A. A small p-value does not establish a business-worthy gain.

Where the journey changes

These are conditional funnel rates. More clicks are not necessarily more buyers.

Click = clicks / visitors; cart = add-to-cart / clicks; checkout = purchases / add-to-cart. Later-stage comparisons involve different, self-selected sets of people and are diagnostic, not separate causal effects.

Across the ten weeks

Weekly purchase rates for A, C and F, calculated from each week’s totals. This checks whether the overall result is confined to a short period.

A · currentC · click leaderF · purchase leader

What to do next

Use purchases per assigned visitor as the launch criterion. A click is an intermediate action; do not ship C solely because its call-to-action attracts attention.
Validate the unit of measurement before a full rollout. Assignment is sticky to a visitor, but the supplied file counts visitors separately each day. If someone appears on multiple days, summed “visitors” are not unique randomized people. Recalculate using one record per assigned visitor, with a fixed purchase-attribution window, and check for duplicate or delayed purchases.
Check business guardrails. This file has no order value, margin, returns, or customer-quality data. Confirm that any purchase gain also improves value and does not harm those measures.
If those checks pass, ramp F gradually while monitoring purchase rate and guardrails against a retained A holdout. Treat this as a decision on the combined F page, not proof that its layout, copy, or call-to-action individually caused the change.

Inference caveat. Random assignment supports comparing versions, but daily aggregates cannot reveal how many distinct people contributed, whether people returned across days, or whether outcomes were fully observed at experiment end. The displayed confidence intervals and tests assume independent visitor records and therefore may be too narrow. They are a screening aid, not a substitute for a unique-visitor analysis.

Source: supplied daily experiment CSV. All charts, totals, and comparisons below are computed locally from the embedded data; no network access is required. A–H received approximately equal traffic. Observational funnel-stage rates are not adjusted for multiple testing.