What is overfitting, how do you detect it, and how do you reduce it?

What the interviewer wants to hear

• Model fits training data noise and does not generalise
• Detected by a gap between training and validation performance
• Fixes: more data, regularisation, simpler model, early stopping
• Use cross-validation to choose complexity

Model answer

Overfitting is when a model learns the training data too closely, including its noise, so it performs well on training data but poorly on new data. I detect it by comparing training and validation metrics: if training error keeps going down while validation error goes up, the model is overfitting. Learning curves make this easy to see. To reduce it, I first try to get more or cleaner data. Then I add regularisation such as L1 or L2 penalties, dropout for neural networks, or limits like max depth for trees. Early stopping based on validation loss is simple and effective. I may also reduce model complexity or features. I use cross-validation to pick these settings so the choice is not tuned to one lucky split, and I keep a separate test set for the final check.

More easy Machine Learning Engineer questions

• Explain precision and recall with a real example. When would you favour one over the other?
• How does gradient descent work, and what is the role of the learning rate?
• What are the training, validation and test sets used for, and how do you split them?
• Why should you build a simple baseline before a complex model? Give examples of good baselines.
• Explain the bias-variance trade-off and how it guides your choice of model.
• What is an embedding, and how would you use embeddings in a product feature?
• When would you choose a tree-based model like gradient boosting over a linear model, and when not?
• How do you handle missing values in a dataset before training a model?
• Explain in simple terms how self-attention works in a transformer.