hard20 pts · first answer countsab-testingexperiment-design

Checking results every day

A product manager checks a fixed-horizon A/B test every day and plans to stop as soon as p < 0.05. What is the problem and the fix?

A.No problem; more checks give more accurate resultsB.The test loses all statistical power after day oneC.Repeated peeking inflates the false positive rate well above 5%; fix the sample size in advance or use a sequential testing method designed for continuous monitoringD.Daily checks only matter for revenue metrics

Coaching the worst stores →