medium10 pts · first answer countsab-testingstakeholder-communication

PM wants to ship an unclear win

A test shows +2% revenue per user, p = 0.20, 95% CI from -1% to +5%. The PM says "it is positive, let us ship". What is the best response?

A.Agree, because the point estimate is positiveB.Refuse, because p > 0.05 proves the change does nothingC.Explain that the data fits anything from a small loss to a solid gain, and decide on risk and cost, or extend the testD.Re-run the analysis with a one-sided test so it becomes significant

← A lift that fades after week one Take the medium mock interview →