A business team does not trust your credit risk model. How do you explain its predictions?
What the interviewer wants to hear
- Global explanations: important features, partial dependence
- Local explanations: SHAP values for individual cases
- Show performance and calibration on recent data
- Check fairness and sensible behaviour with domain experts
- Consider a simpler interpretable model if needed
Model answer
I would explain at two levels. Globally, I show which features matter most and how they affect risk, using SHAP summary plots or partial dependence plots, for example "risk rises steadily with the number of missed payments". I check with credit experts that these patterns make sense. Locally, for individual applicants, SHAP values show which factors pushed a score up or down, which also helps with adverse action reasons customers may be owed. Trust also comes from evidence: I show performance on recent data, calibration, so a predicted 10% default rate really means about 10%, and results across segments to check fairness. I walk through real examples, including mistakes. If the gain from a complex model is small, I would consider a simpler scorecard or logistic model that is easier to explain and govern.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium Data Scientist questions
- Walk through how you would design an A/B test for a new checkout page.
- How would you choose a success metric for a new "recommended for you" section in a shopping app?
- You have 500 candidate features for a churn model. How do you decide which ones to keep?
- The CEO asks: "Why are customers churning?" How do you approach this?
- You are building a model where only 0.5% of examples are positive. How do you approach training and evaluation?
- How would you validate a demand forecasting model before the business relies on it?
- What is data leakage, how do you detect it, and how do you prevent it in a modelling project?
- How do you decide between a regularised linear model and a gradient boosted tree model for a tabular problem?
- Walk me through how you would decide the sample size and duration of an A/B test.