10-Week Randomized Controlled Trial (2026-06-01 to 2026-08-09) • 8 Variants • 1.83M Visitors
Comparison of all 8 candidates from landing to completed purchase.
| Variant | Status | Visitors | Primary Clicks (CTR) | Add-to-Cart (Rate) | Purchases | End-to-End Conv. | Lift vs Baseline (A) | Stat. Significance |
|---|
The primary business metric (Purchases ÷ Total Visitors)
Why CTR alone is a deceptive proxy for business value
Where prospects drop off for Baseline (A), Trap (C), and Winner (F)
Variant F dominated baseline consistently across all 10 weeks
What the visitor behavior reveals about customer intent and page design
Unlike redesigns that used aggressive clickbait buttons, Variant F increased CTA clicks moderately (+8.8%), but critically, those visitors maintained higher add-to-cart rates (+7.9%) and completed purchases at the highest rate in the test (35.7% cart-to-purchase). This proves that Variant F's copy and layout set transparent, accurate product expectations upfront.
Variant C generated a staggering 112,028 clicks (48.7% CTR) and 16,680 cart additions. If the product team had evaluated this test on CTR or Cart Adds, C would have seemed like an astronomical triumph. However, at checkout, conversion plummeted by nearly 29%. Variant C ended with 4,082 purchases (1.78%), which is actually worse than our existing control page A (4,179 purchases / 1.82%). Optimizing for early-funnel clicks cannibalized buyer trust.
Variants B, D, and G performed essentially identical to Baseline A (all within ±1.5% lift, none statistically significant). Variant E suppressed top-of-funnel engagement (-13.5% CTR) due to low-contrast or misplaced CTA. Variant H was deeply counterproductive (-14.7% total purchases, p < 0.0001), indicating confusing layout or friction.
Immediate operational steps for product, design, and engineering
Variant F is thoroughly de-risked across 228,784 visitors and 70 consecutive days. Phasing out A through H and serving F to all traffic will capture an estimated ~4,800 incremental orders annually on current run-rates.
Have the UX/copywriting team conduct a side-by-side audit of C and F. Variant C possessed an exceptional hook (generating ~31,000 extra clicks) while Variant F possessed superior qualification and checkout follow-through. Our next hypothesis: test whether F's layout with C's headline framing can yield an even stronger hybrid page.
Variant H and E introduced measurable conversion blockers; their copy/layout patterns should be documented in the design system as anti-patterns to avoid in future iterations.
This experiment highlighted the critical risk of judging tests on micro-metrics like CTA clicks. Enforce a company-wide analytics guideline: no landing page experiment may graduate without statistical significance on verified down-funnel orders or revenue.