hard20 pts · first answer countsab-testingstatistics

Stopping an A/B test too early

Two days into a planned 3-week landing page test, the new version shows a 20% lift based on 40 total conversions. Your stakeholder wants to declare a winner now. What do you advise?

A.Stop now; a 20% lift is large enough to be realB.Continue to the planned sample size and at least one or two full weekly cycles, because early results with few conversions are often noiseC.Stop the test and run the old page, because tests are unreliableD.Keep checking every hour and stop as soon as significance is reached

← Measurement with privacy signal loss Defend the brand budget →