Design an automated retraining pipeline for a model whose data changes weekly.

What the interviewer wants to hear

• Scheduled or triggered runs via an orchestrator
• Data extraction with validation checks
• Training with tracked config and versioned data
• Evaluation against production model and slices
• Registry promotion and gradual rollout; alerts on failure

Model answer

I would use an orchestrator like Airflow or Kubeflow Pipelines. A weekly schedule starts the pipeline, and drift or performance alerts can also trigger it. Step one extracts a fixed time window of data and runs validation: schema, row counts, missing values and label distribution. If validation fails, the pipeline stops and alerts, since training on broken data is worse than not training. Step two trains with a versioned config and logs everything to the experiment tracker. Step three evaluates the new model against the current production model on a recent holdout, overall and by important segments, plus latency checks. If it wins by a set margin and passes all gates, it is registered and promoted, then deployed by canary. Every step is idempotent so retries are safe, and failures notify the owning team.

More medium MLOps Engineer questions

• What does a good CI/CD pipeline for machine learning look like?
• Explain shadow, canary and A/B deployments for models. When do you use each?
• What operational challenges come with running a feature store, and how do you handle them?
• How do you run GPU model inference efficiently on Kubernetes?
• How do you design a training pipeline in an orchestrator like Airflow so that it is reliable and easy to rerun?
• How would you estimate the GPUs needed to serve an open-weight LLM for a given number of users?
• How would you define service level objectives for a production model service?
• How do you secure the ML pipeline and model supply chain?
• What would you monitor for an LLM-powered application in production, beyond normal service metrics?