Design a real-time fraud detection system for card payments.

What the interviewer wants to hear

• Extreme class imbalance and costly errors in both directions
• Real-time features such as velocity counts from a streaming feature store
• Low-latency model, often gradient boosting, plus rules
• Thresholds set from business cost, not accuracy
• Delayed labels, adversarial drift and frequent retraining
• Time-based validation and shadow testing before rollout

Model answer

Fraud is rare, often well under 1% of transactions, and both missed fraud and blocked good customers are costly, so I optimise expected cost and use precision-recall metrics. The decision must happen in tens of milliseconds. Strong signals are behavioural, such as how many transactions a card made in the last hour, new device, or distance from usual locations, so I compute these in a streaming feature store that serves the same definitions online and offline. A gradient-boosted model scores each transaction, combined with hard rules for known patterns, and thresholds map scores to approve, challenge or block based on cost. Labels arrive late through chargebacks, and fraudsters adapt, so I validate on later time periods, retrain often and monitor score distributions and fraud rates. New models run in shadow mode before taking over.

More hard Machine Learning Engineer questions

• Design a recommendation system for an e-commerce home page with 10 million products and 50 million users.
• A model performs much worse in production than in offline tests. How do you find and prevent training-serving skew?
• You need to train a model that is too large for one GPU and takes weeks on one machine. How do you scale training?
• How would you run an A/B test to decide whether a new ML model should replace the current one? What pitfalls do you watch for?
• A deep learning model meets accuracy targets but is too slow and expensive to serve. What options do you consider?
• How would you design LLM inference serving to maximise throughput while keeping latency acceptable?
• What guardrails would you put around an LLM agent that can call internal tools and read customer data?
• Scaling training from 8 to 64 GPUs gave only a 3x speed-up. How do you find and fix the bottleneck?
• You need to cut the serving cost of a large model by quantising it. How do you do it without hurting quality?