Explain retrieval-augmented generation (RAG) to a new team member. Why do we use it?

What the interviewer wants to hear

• Retrieve relevant documents for the question
• Add them to the prompt as context
• Model answers grounded in the context, can cite sources
• Works for private and fresh data without retraining

Model answer

RAG means we look things up before the model answers. When a user asks a question, we search our own knowledge base, usually by turning the question into an embedding and finding similar document chunks in a vector database, sometimes combined with keyword search. We take the best few chunks and put them into the prompt with instructions like "answer only using these sources and cite them". The model then writes an answer grounded in that context. We use it because the base model does not know our private documents and its knowledge has a cutoff date. RAG lets us use fresh and private data without retraining the model, reduces hallucinations, and lets users check citations. The quality of the answer depends heavily on the quality of retrieval, so we measure that separately.

More easy AI Engineer (LLM & Generative AI) questions

• What are embeddings, and what can you build with them besides RAG?
• Why do LLMs hallucinate, and what simple steps reduce it in an application?
• How do tokens affect the cost and speed of an LLM application?
• What do temperature and top-p control, and how would you set them for different tasks?
• What is an LLM agent, and how is it different from a single LLM call?
• How does vector search work, and why is it useful for an AI application?
• Your app starts getting rate-limit errors from the LLM API. How do you handle them?
• What is the difference between zero-shot and few-shot prompting, and when would you use each?
• How do you protect personal data when building a feature on a hosted LLM API?