medium10 pts · first answer countsmodel-evaluationdata-splittinghyperparameter-tuning
Hundreds of runs on one validation set
A team has tried 400 model variants, always picking by the same validation set. The final test score is much lower than the best validation score. What is the likely cause?
Discussion 0
Join free to post in the discussion.
Related questions
- Big gap between train and validation accuracyMachine Learning Engineer
- Why keep a separate test setMachine Learning Engineer
- How often a row is validated in k-foldMachine Learning Engineer
- Why stratify the foldsMachine Learning Engineer
- Specificity by handMachine Learning Engineer
- The x-axis of an ROC curveMachine Learning Engineer
- MAE of three errorsMachine Learning Engineer
- MAPE by handMachine Learning Engineer