A high-volume prompt costs too much. How do you reduce cost while protecting quality?

What the interviewer wants to hear

• Measure token usage and cost per request
• Trim instructions and examples; dynamic example selection
• Prompt caching for static prefixes
• Route easy cases to smaller models
• Limit output length; validate quality with evals

Model answer

First I measure input and output tokens per request to find what drives cost. Then I trim: remove repeated or unnecessary instructions, shorten examples, and only include the few examples most relevant to each input instead of a fixed long list. I order the prompt so the static part comes first and benefits from prompt caching. I control output length with clear limits, since output tokens are often the more expensive part. Next I try a smaller, cheaper model, often with a slightly adjusted prompt, and route only the hard cases to the large model, using simple rules or a classifier. For non-urgent work, batch processing can be cheaper. Every change is tested against the eval set, and I present the cost savings next to any quality change so the business can choose the right balance.

More hard Prompt Engineer questions

• What is prompt injection, and how would you reduce the risk for an assistant that reads emails and can draft and send replies?
• Design an LLM-as-judge setup to evaluate a summarisation feature. How do you make it trustworthy?
• Your company is moving its prompts from one model provider to another. How do you plan and run the migration?
• You need an assistant that serves users writing in Hindi, Tamil and mixed Hinglish. What prompting and evaluation challenges do you expect?
• You are building an agent that reads emails and web pages and can send messages and update a CRM. How do you protect it from indirect prompt injection?
• How would you evaluate a multi-step agent that completes tasks using tools, rather than a single prompt?
• How do you show with confidence that a new prompt is better than the current one, and not just lucky on your test set?
• Design a layered guardrail system for a public-facing LLM assistant. What goes where, and what are the trade-offs?
• An agent runs long tasks that exceed its context window. How do you manage context so quality stays high?