hard20 pts · first answer countsmodel-evaluationstatisticsab-testing
Twenty metrics, one significant
An A/B test checks 20 metrics, each at p < 0.05. One metric shows a "significant" lift and nothing else moved. Why be careful?
Discussion 0
Join free to post in the discussion.
Related questions
- Big gap between train and validation accuracyMachine Learning Engineer
- Why keep a separate test setMachine Learning Engineer
- Standard error of a sample meanMachine Learning Engineer
- How often a row is validated in k-foldMachine Learning Engineer
- Why stratify the foldsMachine Learning Engineer
- Specificity by handMachine Learning Engineer
- The x-axis of an ROC curveMachine Learning Engineer
- MAE of three errorsMachine Learning Engineer