How do you design good tools (functions) for an LLM to call?
What the interviewer wants to hear
- Clear names, descriptions and typed parameters
- Few focused tools rather than many overlapping ones
- Return concise, useful results and helpful error messages
- Validate arguments and enforce permissions in code
- Make actions idempotent and confirm risky ones
Model answer
The model only knows what the tool name, description and schema tell it, so I write them like documentation for a new developer: what the tool does, when to use it, and what each parameter means, with types and enums. I prefer a small set of focused tools over many overlapping ones, because overlap confuses the model. Tool results should be concise and relevant, not huge raw dumps that waste context, and errors should explain what went wrong so the model can fix its call. On the application side I validate every argument, enforce the user permissions in code rather than trusting the model, and follow least privilege. For actions with side effects, like refunds or sending emails, I make them idempotent and require confirmation. I test tool selection with evals that check the right tool and arguments are chosen.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium AI Engineer (LLM & Generative AI) questions
- How do you decide between better prompting, RAG and fine-tuning for a new use case?
- How do you evaluate an LLM-powered feature before and after launch?
- Your RAG bot often answers "I could not find this" even though the answer is in the documents. How do you improve retrieval?
- You need an LLM to extract fields from invoices into your database. How do you make this reliable?
- How do you measure the retrieval part of a RAG system separately from the generation part?
- How would you implement streaming LLM responses in a web app, and what problems do you need to handle?
- What kinds of caching can you use in an LLM application, and what are the risks of each?
- What would you put in place to observe and monitor an LLM application in production?
- How would you build and maintain a golden evaluation set for an LLM feature?