What makes a good Docker image for a model-serving service?

What the interviewer wants to hear

• Small, pinned base image and pinned dependencies
• Layer order that caches dependency installs
• No secrets or training data baked in
• Runs as a non-root user with a health endpoint
• Model loaded from a versioned registry or included with a recorded version

Model answer

I start from a small, pinned base image, such as a slim Python image with a specific version, and install pinned dependencies from a lock file. I copy the requirements file and install packages before copying the application code, so code changes do not reinstall everything. If packages need compilers, I use a multi-stage build so the final image only contains what runs. I never bake secrets or training data into the image; secrets are injected at runtime. The container runs as a non-root user and exposes health and readiness endpoints. For the model itself, I either download a specific version from the registry at startup or include it in the image and record the version, so every running container can be traced to an exact model. Finally I scan the image for vulnerabilities in CI and keep it as small as practical so it starts quickly.

More easy MLOps Engineer questions

• What is MLOps, and how is it different from traditional DevOps?
• Why does a team need a model registry? What information should it store?
• How do you make a model training run reproducible?
• How would you deploy a trained scikit-learn model as a REST API?
• What would you monitor for a model running in production?
• What is training-serving skew, and how do you prevent it?
• How do you decide between batch predictions and a real-time model API?
• Explain the difference between data drift and concept drift, and how you would detect each.
• Which Kubernetes concepts matter most when you deploy a model service, and why?