Why should you build a simple baseline before a complex model? Give examples of good baselines.

What the interviewer wants to hear

• Baseline sets a reference point to measure real improvement
• Catches bugs and data problems early
• Examples: majority class, mean prediction, logistic regression, simple rules
• Complexity must justify its cost

Model answer

A baseline tells me what "good" looks like. Without it I cannot say whether a fancy model is actually helping. For classification, the simplest baseline is predicting the majority class; for regression it is predicting the mean or the last value. A step up is a simple model such as logistic regression or a small gradient boosted tree, or a business rule the team already uses. Baselines also help catch bugs: if my deep model barely beats predicting the mean, I probably have a data or pipeline problem. Finally, complex models cost more to train, serve and maintain, and are harder to explain. If a simple model gets 95% of the value, it is often the right production choice. I always report new results against the baseline on the same validation data.

More easy Machine Learning Engineer questions

• What is overfitting, how do you detect it, and how do you reduce it?
• Explain precision and recall with a real example. When would you favour one over the other?
• How does gradient descent work, and what is the role of the learning rate?
• What are the training, validation and test sets used for, and how do you split them?
• Explain the bias-variance trade-off and how it guides your choice of model.
• What is an embedding, and how would you use embeddings in a product feature?
• When would you choose a tree-based model like gradient boosting over a linear model, and when not?
• How do you handle missing values in a dataset before training a model?
• Explain in simple terms how self-attention works in a transformer.