What guardrails would you put around an LLM agent that can call internal tools and read customer data?

What the interviewer wants to hear

• Least-privilege tools and scoped credentials per user
• Treat user and retrieved content as untrusted; defend against prompt injection
• Input and output filters for harmful content and data leaks
• Human confirmation for high-impact actions
• Logging, red teaming and continuous evaluation of safety cases

Model answer

I assume the model can be manipulated, so safety cannot rely on the prompt alone. Each tool gets the narrowest permissions possible, and calls run with the end user's credentials, so the agent can never see more data than that user can. Retrieved documents, emails and web pages are treated as untrusted data that may contain injected instructions. I validate tool arguments against schemas and allow-lists, and actions with real impact, such as sending money, emailing outside the company or deleting records, need explicit human confirmation. Input and output filters check for harmful content, secrets and personal data before anything leaves the system. Everything is logged for audit. Before launch we red team the agent with injection and jailbreak attempts, turn those into a regression suite, and keep running it on every model or prompt change.

More hard Machine Learning Engineer questions

• Design a recommendation system for an e-commerce home page with 10 million products and 50 million users.
• A model performs much worse in production than in offline tests. How do you find and prevent training-serving skew?
• You need to train a model that is too large for one GPU and takes weeks on one machine. How do you scale training?
• How would you run an A/B test to decide whether a new ML model should replace the current one? What pitfalls do you watch for?
• A deep learning model meets accuracy targets but is too slow and expensive to serve. What options do you consider?
• How would you design LLM inference serving to maximise throughput while keeping latency acceptable?
• Scaling training from 8 to 64 GPUs gave only a 3x speed-up. How do you find and fix the bottleneck?
• Design a real-time fraud detection system for card payments.
• You need to cut the serving cost of a large model by quantising it. How do you do it without hurting quality?