When would you split a task into a chain of prompts instead of one large prompt? What are the trade-offs?

What the interviewer wants to hear

• Chain when task has distinct steps or skills
• Easier to test, debug and improve each step
• Structured handoffs between steps
• Trade-offs: more latency, more cost, error propagation
• Single prompt is simpler when it performs well enough

Model answer

I split a task when it has clearly different steps, like extracting facts, then checking them against rules, then writing a message. Each smaller prompt can focus on one job, which usually improves quality. It is also easier to evaluate and debug: if the final email is wrong, I can see whether extraction or drafting failed. I pass structured outputs between steps so they can be validated in code, and some steps might not even need a model. The trade-offs are more calls, so more latency and cost, and errors from early steps flow into later ones, so the first steps need to be reliable. If a single prompt already meets the quality bar on the eval set, I keep it, because simple systems are easier to maintain.

More medium Prompt Engineer questions

• How would you build an evaluation set for a prompt that writes replies to customer complaints?
• A marketing team wants all generated content to match their brand voice. How do you achieve consistency?
• How do you know when prompt engineering has reached its limit and another approach, such as RAG or fine-tuning, is needed?
• Write the outline of a system prompt for a banking support assistant. What sections would it include and why?
• Design the prompt and retrieval setup for a customer support bot that answers from a help centre. What choices matter most?
• How do you design tool definitions so an LLM agent uses them correctly?
• Your LLM feature works well but is too slow and expensive. How would you reduce cost and latency without hurting quality?
• How would you handle personal data in an LLM workflow that processes customer emails?
• How would you set up regression testing for prompts so changes can be shipped safely?