Explain shadow, canary and A/B deployments for models. When do you use each?

What the interviewer wants to hear

• Shadow: new model gets copy of traffic, no user impact
• Canary: small traffic share to check health, gradual ramp
• A/B test: measure business impact with statistics
• Often used in sequence
• Need clear metrics and rollback criteria

Model answer

In a shadow deployment, the new model receives a copy of live requests but its outputs are only logged, so users are not affected. I use it to check latency, errors and how predictions differ from the current model on real data. In a canary, a small share of real traffic, say 5%, is served by the new model; if health and key metrics stay good, I ramp up gradually. It is mainly a safety check. An A/B test splits users randomly for long enough to measure business impact, like conversion, with statistical confidence. I often use all three in order: shadow to catch technical issues, canary for safe exposure, and A/B to prove value. Each stage needs clear success and rollback criteria decided in advance, and automation to roll back fast.

More medium MLOps Engineer questions

• What does a good CI/CD pipeline for machine learning look like?
• Design an automated retraining pipeline for a model whose data changes weekly.
• What operational challenges come with running a feature store, and how do you handle them?
• How do you run GPU model inference efficiently on Kubernetes?
• How do you design a training pipeline in an orchestrator like Airflow so that it is reliable and easy to rerun?
• How would you estimate the GPUs needed to serve an open-weight LLM for a given number of users?
• How would you define service level objectives for a production model service?
• How do you secure the ML pipeline and model supply chain?
• What would you monitor for an LLM-powered application in production, beyond normal service metrics?