When would you choose a tree-based model like gradient boosting over a linear model, and when not?
What the interviewer wants to hear
- Trees capture non-linear effects and interactions automatically
- Trees need little preprocessing and no feature scaling
- Linear models are simpler, faster and easier to explain
- Linear models extrapolate; trees cannot predict outside the training range
- Choose by validation results plus latency and interpretability needs
Model answer
For tabular data with mixed features, gradient-boosted trees are often my first strong model. They find non-linear relationships and feature interactions on their own, handle missing values, and do not need feature scaling. They usually win on accuracy for this kind of data. I would pick a linear or logistic regression when the relationship is close to linear, when I need very clear explanations, for example in regulated credit decisions, when data is small, or when latency and model size must be tiny. Linear models also extrapolate, while trees predict flat values beyond the range they saw in training, which matters for trends over time. In practice I start with a simple linear baseline, then try boosting, and choose using cross-validated metrics together with interpretability and serving cost.
Can you answer it under time pressure?
Take a timed easy round and get scored on every answer.
More easy Machine Learning Engineer questions
- What is overfitting, how do you detect it, and how do you reduce it?
- Explain precision and recall with a real example. When would you favour one over the other?
- How does gradient descent work, and what is the role of the learning rate?
- What are the training, validation and test sets used for, and how do you split them?
- Why should you build a simple baseline before a complex model? Give examples of good baselines.
- Explain the bias-variance trade-off and how it guides your choice of model.
- What is an embedding, and how would you use embeddings in a product feature?
- How do you handle missing values in a dataset before training a model?
- Explain in simple terms how self-attention works in a transformer.