hard20 pts · first answer countsab-testingstatistics

Stopping a test early

A PM checks the A/B test every day and plans to stop the moment p < 0.05. Why is this a problem with a standard fixed-horizon test?

A.It is not a problem; checking more often makes the result more accurateB.Daily checking reduces the sample sizeC.Repeated checking with an early stop greatly increases the false positive rate above 5%D.p-values can only be calculated once per test

← Overall rate goes the other way NOT IN returns nothing →