An agent that completes multi-step tasks works in demos but fails often in production. How do you make it reliable?

What the interviewer wants to hear

• Prefer simple workflows; use agents only where needed
• Tracing to find failure steps
• Better tools, clearer instructions, structured state
• Budgets, retries, checkpoints, human-in-the-loop
• Task-level evals with realistic scenarios

Model answer

First I add tracing so I can see each step: the prompt, the tool calls, the results and where it went wrong. Usually failures cluster: wrong tool choice, bad arguments, confusing tool output, or losing track of the goal. Then I simplify. Many tasks do not need a free-form agent; a fixed workflow with LLM steps at specific points is more predictable, so I only keep autonomy where it adds value. I improve tool descriptions and errors, keep explicit state like a task plan or checklist, and validate each step output. I add hard limits on steps, tokens and time, retries for transient tool errors, and checkpoints so long tasks can resume. Risky actions get human approval. Finally I build task-level evals from real production failures and measure success rate, cost and steps per task on every change.

More hard AI Engineer (LLM & Generative AI) questions

• Design a RAG assistant over 2 million internal documents with different access permissions per team.
• How would you protect an LLM application that reads emails and can take actions, against prompt injection and data leaks?
• Your LLM feature costs too much and p95 latency is 9 seconds. Walk through how you would reduce both without hurting quality.
• When would you fine-tune a smaller open-weight model instead of using a large hosted model, and how would you do it?
• You must self-host an open-weights LLM for a high-traffic product. How do you plan memory, throughput and latency?
• Users say your AI assistant got worse this week, but nothing obvious changed. How do you investigate?
• How do you design an LLM-as-judge evaluation you can actually trust?
• When would you choose a multi-agent architecture over a single agent or a fixed workflow?
• Your agent will connect to several MCP servers, some built by third parties. How do you make this secure?