How would you define service level objectives for a production model service?

What the interviewer wants to hear

• Availability and latency SLOs based on user needs
• Model quality indicators such as prediction drift or delayed accuracy
• Data freshness SLOs for features and batch outputs
• Error budgets that guide release pace
• Alerts on symptoms users feel

Model answer

I start from what users of the model need. For an online service the basic indicators are availability, measured as successful responses, and latency, usually p95 or p99 against a target such as 150 ms. For ML I add indicators that normal services lack: freshness of features and of batch predictions, for example scores updated by 6 am every day, and quality signals such as prediction distribution stability and accuracy on delayed labels. Each SLO has a target and an error budget, such as 99.9% availability over 30 days, which is about 43 minutes of downtime. If we burn the budget quickly, we slow down risky changes like model swaps. Alerts fire on budget burn rate and user-facing symptoms rather than on every low-level metric, so on-call engineers are not flooded with noise.

More medium MLOps Engineer questions

• What does a good CI/CD pipeline for machine learning look like?
• Explain shadow, canary and A/B deployments for models. When do you use each?
• Design an automated retraining pipeline for a model whose data changes weekly.
• What operational challenges come with running a feature store, and how do you handle them?
• How do you run GPU model inference efficiently on Kubernetes?
• How do you design a training pipeline in an orchestrator like Airflow so that it is reliable and easy to rerun?
• How would you estimate the GPUs needed to serve an open-weight LLM for a given number of users?
• How do you secure the ML pipeline and model supply chain?
• What would you monitor for an LLM-powered application in production, beyond normal service metrics?