Skip to content
RED SAGSkills
mediumData Scientist interview question

What is data leakage, how do you detect it, and how do you prevent it in a modelling project?

What the interviewer wants to hear

  • Leakage is information in training that will not exist at prediction time
  • Signs include suspiciously high scores and a single dominant feature
  • Common causes: future data, label-derived features, preprocessing on all data, duplicate entities across splits
  • Prevent with point-in-time features, pipelines and proper splits

Model answer

Leakage is when the model learns from information that will not be available when it makes real predictions, so offline scores look great but production results do not. Warning signs are scores that seem too good, like 0.99 AUC on a hard problem, or one feature dominating importance. I investigate those features and ask when each value becomes known. Common causes are features derived from the label, such as a refund flag in a fraud model, using future data in time series, fitting scalers or encoders on the whole dataset before splitting, and the same customer appearing in both train and test. To prevent it I build features with point-in-time logic using an as-of timestamp, put all preprocessing inside a pipeline that is fitted on training folds only, split by time or entity when needed, and finally compare offline metrics with a short shadow run in production.

Can you answer it under time pressure?

Take a timed medium round and get scored on every answer.

Start mock interview

More medium Data Scientist questions