How do you decide between a regularised linear model and a gradient boosted tree model for a tabular problem?

What the interviewer wants to hear

• Linear models are interpretable, stable and good with few rows or many sparse features
• Boosted trees capture non-linearity and interactions with little feature work
• Consider explainability, latency and maintenance needs
• Compare both against a baseline with proper validation

Model answer

I usually try both, because the comparison itself is informative. A regularised linear or logistic model is fast, stable, easy to explain with coefficients and works well when data is small, very wide or sparse, such as text features. It needs manual work for non-linear effects and interactions. Gradient boosted trees usually win on accuracy for medium to large tabular data because they find non-linearities and interactions automatically and handle missing values and mixed scales. The trade-offs are less transparent explanations, more tuning and a higher risk of overfitting on small data. I would also weigh the context: if a regulator needs clear reasons, or the gain from boosting is tiny, I prefer the linear model. I compare them on the same validation scheme against a simple baseline, and look at calibration and stability across time periods, not only the headline metric.

More medium Data Scientist questions

• Walk through how you would design an A/B test for a new checkout page.
• How would you choose a success metric for a new "recommended for you" section in a shopping app?
• You have 500 candidate features for a churn model. How do you decide which ones to keep?
• A business team does not trust your credit risk model. How do you explain its predictions?
• The CEO asks: "Why are customers churning?" How do you approach this?
• You are building a model where only 0.5% of examples are positive. How do you approach training and evaluation?
• How would you validate a demand forecasting model before the business relies on it?
• What is data leakage, how do you detect it, and how do you prevent it in a modelling project?
• Walk me through how you would decide the sample size and duration of an A/B test.