hard20 pts · first answer countsab-testingstatistics

Click-through rate standard errors

You randomise by user but compute click-through rate as total clicks divided by total page views, then use a simple two-proportion test that treats every page view as independent. What is wrong?

A.Click-through rate cannot be used in experimentsB.Page views from the same user are correlated, so the naive test underestimates variance; use the delta method or a user-level bootstrapC.The test overestimates variance, so results are too conservativeD.You must randomise by page view to compute any metric

← Feature users retain better Propensity scores near 0 and 1 →