What is training-serving skew, and how do you prevent it?

What the interviewer wants to hear

• Model sees different feature values in production than in training
• Caused by duplicated logic, different data sources or timing
• Share one feature definition or package transforms with the model
• Log served features and compare with offline values

Model answer

Training-serving skew happens when the features a model receives in production are not computed the same way as the features it was trained on. A common cause is writing the logic twice, for example in SQL for training and in application code for serving, and the two drift apart in rounding, time zones or how missing values are filled. It can also come from using different data sources or from timing, such as using data in training that is not yet available at request time. To prevent it, I keep a single feature definition, either in a feature store or by packaging the preprocessing inside the model artifact, so both paths run the same code. I also log the features actually used at serving time and run a regular parity check that compares them with what the offline pipeline would produce for the same entities.

More easy MLOps Engineer questions

• What is MLOps, and how is it different from traditional DevOps?
• Why does a team need a model registry? What information should it store?
• How do you make a model training run reproducible?
• How would you deploy a trained scikit-learn model as a REST API?
• What would you monitor for a model running in production?
• How do you decide between batch predictions and a real-time model API?
• Explain the difference between data drift and concept drift, and how you would detect each.
• What makes a good Docker image for a model-serving service?
• Which Kubernetes concepts matter most when you deploy a model service, and why?