What operational challenges come with running a feature store, and how do you handle them?
What the interviewer wants to hear
- Keeping online and offline stores consistent
- Freshness and backfills
- Point-in-time correctness
- Latency and availability of online store
- Ownership, documentation and cost of unused features
Model answer
The biggest challenge is consistency between offline and online stores. I make both come from the same feature definitions and materialisation jobs, and run regular checks that compare samples from both. Freshness is next: each feature needs an agreed update frequency and alerts when materialisation jobs are late. Backfills are needed when a feature changes or is added, and they must be point-in-time correct so training does not leak future data. The online store must be fast and highly available because every prediction depends on it, so I set latency targets, replicate it, and have fallbacks or default values if a lookup fails. Finally, governance: every feature needs an owner and documentation, and I track usage so unused features can be removed instead of costing compute forever.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium MLOps Engineer questions
- What does a good CI/CD pipeline for machine learning look like?
- Explain shadow, canary and A/B deployments for models. When do you use each?
- Design an automated retraining pipeline for a model whose data changes weekly.
- How do you run GPU model inference efficiently on Kubernetes?
- How do you design a training pipeline in an orchestrator like Airflow so that it is reliable and easy to rerun?
- How would you estimate the GPUs needed to serve an open-weight LLM for a given number of users?
- How would you define service level objectives for a production model service?
- How do you secure the ML pipeline and model supply chain?
- What would you monitor for an LLM-powered application in production, beyond normal service metrics?