Design a recommendation system for an e-commerce home page with 10 million products and 50 million users.

What the interviewer wants to hear

• Multi-stage design: candidate generation then ranking then re-ranking
• Two-tower embeddings and ANN search for retrieval
• Ranking model with rich features, trained on engagement and conversions
• Cold start, diversity and business rules
• Offline metrics plus online A/B tests; latency budget

Model answer

I would use a multi-stage system. First, candidate generation picks a few hundred products from 10 million within tens of milliseconds. A two-tower model learns user and product embeddings, and I retrieve neighbours with an approximate nearest neighbour index, alongside simple sources like trending items and recently viewed categories. Second, a ranking model, such as gradient boosting or a deep model, scores candidates using user history, product features, price, context and cross features, trained on clicks and purchases with weights favouring conversions. Third, re-ranking adds diversity, removes out-of-stock items and applies business rules. For cold start I use content features and popularity for new products and new users. Features come from a feature store to avoid skew. I evaluate offline with recall and NDCG, but trust online A/B tests on conversion and revenue, keeping the total latency under around 150 milliseconds.

More hard Machine Learning Engineer questions

• A model performs much worse in production than in offline tests. How do you find and prevent training-serving skew?
• You need to train a model that is too large for one GPU and takes weeks on one machine. How do you scale training?
• How would you run an A/B test to decide whether a new ML model should replace the current one? What pitfalls do you watch for?
• A deep learning model meets accuracy targets but is too slow and expensive to serve. What options do you consider?
• How would you design LLM inference serving to maximise throughput while keeping latency acceptable?
• What guardrails would you put around an LLM agent that can call internal tools and read customer data?
• Scaling training from 8 to 64 GPUs gave only a 3x speed-up. How do you find and fix the bottleneck?
• Design a real-time fraud detection system for card payments.
• You need to cut the serving cost of a large model by quantising it. How do you do it without hurting quality?