Explain the bias-variance trade-off and how it guides your choice of model.

What the interviewer wants to hear

• Bias is error from overly simple assumptions (underfitting)
• Variance is sensitivity to the particular training sample (overfitting)
• More complexity lowers bias but raises variance
• Use validation curves or cross-validation to find the sweet spot
• Regularisation, more data and ensembles shift the balance

Model answer

Bias is the error that comes from a model being too simple to capture the real pattern, like fitting a straight line to a curve. It shows up as poor scores on both training and validation data, which is underfitting. Variance is how much the model changes when trained on a different sample. High variance means it memorises noise, so training scores are great but validation scores are poor, which is overfitting. As model complexity rises, bias usually falls and variance rises, so total error is lowest somewhere in the middle. I find that point with learning curves and cross-validation. To reduce variance I add data, regularisation such as L2 or dropout, or use bagging. To reduce bias I add features, use a more flexible model or train longer.

More easy Machine Learning Engineer questions

• What is overfitting, how do you detect it, and how do you reduce it?
• Explain precision and recall with a real example. When would you favour one over the other?
• How does gradient descent work, and what is the role of the learning rate?
• What are the training, validation and test sets used for, and how do you split them?
• Why should you build a simple baseline before a complex model? Give examples of good baselines.
• What is an embedding, and how would you use embeddings in a product feature?
• When would you choose a tree-based model like gradient boosting over a linear model, and when not?
• How do you handle missing values in a dataset before training a model?
• Explain in simple terms how self-attention works in a transformer.