You need an LLM to extract fields from invoices into your database. How do you make this reliable?

What the interviewer wants to hear

• Define a strict schema with types and enums
• Use structured output or tool calling
• Validate with code; business rule checks
• Retry or route to human review on failure or low confidence
• Evaluate field-level accuracy on labelled invoices

Model answer

I define a clear schema: field names, types, enums for things like currency, and nullable fields so the model can say a value is missing instead of inventing it. I use the provider structured output or tool calling so responses follow the schema. Then I validate in code with something like Pydantic or Zod, and add business checks: line items should add up to the total, dates should be valid, the tax ID should match a pattern. If validation fails, I retry with the error message included, and if it still fails or values look suspicious, I send the invoice to a human review queue. Low temperature helps consistency. Before launch I build a labelled set of real invoices and measure accuracy per field, and I keep monitoring correction rates from reviewers after launch.

More medium AI Engineer (LLM & Generative AI) questions

• How do you decide between better prompting, RAG and fine-tuning for a new use case?
• How do you evaluate an LLM-powered feature before and after launch?
• Your RAG bot often answers "I could not find this" even though the answer is in the documents. How do you improve retrieval?
• How do you design good tools (functions) for an LLM to call?
• How do you measure the retrieval part of a RAG system separately from the generation part?
• How would you implement streaming LLM responses in a web app, and what problems do you need to handle?
• What kinds of caching can you use in an LLM application, and what are the risks of each?
• What would you put in place to observe and monitor an LLM application in production?
• How would you build and maintain a golden evaluation set for an LLM feature?