Which Kubernetes concepts matter most when you deploy a model service, and why?

What the interviewer wants to hear

• Deployments and replicas for scaling and rolling updates
• Services for stable networking and load balancing
• Resource requests and limits for CPU, memory and GPU
• Readiness and liveness probes
• ConfigMaps and Secrets for configuration

Model answer

The core object is a Deployment, which runs a set number of identical pods and handles rolling updates, so a new model version can replace the old one gradually. A Service gives those pods a stable address and load-balances between them. Resource requests and limits matter a lot for models: memory requests must cover model loading, and GPU workloads request GPUs explicitly so they land on the right nodes. Readiness probes stop traffic reaching a pod until the model is loaded, and liveness probes restart pods that hang, but with a long enough start delay. ConfigMaps and Secrets keep configuration and credentials out of the image. Finally, a Horizontal Pod Autoscaler adds replicas under load, ideally scaling on request rate or queue depth rather than CPU for GPU services.

More easy MLOps Engineer questions

• What is MLOps, and how is it different from traditional DevOps?
• Why does a team need a model registry? What information should it store?
• How do you make a model training run reproducible?
• How would you deploy a trained scikit-learn model as a REST API?
• What would you monitor for a model running in production?
• What is training-serving skew, and how do you prevent it?
• How do you decide between batch predictions and a real-time model API?
• Explain the difference between data drift and concept drift, and how you would detect each.
• What makes a good Docker image for a model-serving service?