Design a layered guardrail system for a public-facing LLM assistant. What goes where, and what are the trade-offs?
What the interviewer wants to hear
- Input checks for abuse, jailbreaks and PII
- Clear system prompt policy and scoped tools
- Output checks for policy, PII and grounding
- Fallbacks, escalation and human review
- Trade-offs in latency, cost and false positives
- Monitoring and eval of the guardrails themselves
Model answer
I would use defence in depth, because each layer misses things. At input, fast classifiers catch abuse, jailbreak patterns and sensitive data, and rate limits slow attackers. The system prompt states the scope and policies clearly, and tools are limited to what the assistant needs. At output, checks look for policy violations, leaked personal data and, for factual answers, claims not supported by retrieved sources. When a check fails, the app refuses politely, rewrites, or escalates to a human, depending on severity. The trade-offs are real: each extra model call adds latency and cost, so I use small, fast checkers and run some in parallel. Too-strict filters frustrate legitimate users, so I measure false positives as well as misses on labelled test sets and review flagged traffic regularly.
Can you answer it under time pressure?
Take a timed hard round and get scored on every answer.
More hard Prompt Engineer questions
- What is prompt injection, and how would you reduce the risk for an assistant that reads emails and can draft and send replies?
- Design an LLM-as-judge setup to evaluate a summarisation feature. How do you make it trustworthy?
- Your company is moving its prompts from one model provider to another. How do you plan and run the migration?
- You need an assistant that serves users writing in Hindi, Tamil and mixed Hinglish. What prompting and evaluation challenges do you expect?
- A high-volume prompt costs too much. How do you reduce cost while protecting quality?
- You are building an agent that reads emails and web pages and can send messages and update a CRM. How do you protect it from indirect prompt injection?
- How would you evaluate a multi-step agent that completes tasks using tools, rather than a single prompt?
- How do you show with confidence that a new prompt is better than the current one, and not just lucky on your test set?
- An agent runs long tasks that exceed its context window. How do you manage context so quality stays high?