Explain retrieval-augmented generation (RAG) to a non-technical colleague. Why would we use it?

What the interviewer wants to hear

• Search relevant documents first, then answer from them
• Gives the model private or up-to-date knowledge
• Reduces hallucination and allows citations
• Cheaper and faster to update than retraining

Model answer

I would say RAG is like an open-book exam for the AI. A model only knows what it learned during training, so it does not know our internal policies or anything that changed last week. With RAG, when someone asks a question, we first search our own documents, such as help articles or policy files, and pick the most relevant passages. We put those passages in the prompt and ask the model to answer using them. This has three benefits. Answers use our real, current information. The model makes up less, especially if we tell it to say when the documents do not cover the question. And we can show which document each answer came from. Updating knowledge is easy, because we just update the documents rather than retraining a model.

More easy Prompt Engineer questions

• What makes a good prompt? Walk me through how you would write one for a new task.
• When would you use zero-shot prompting and when would you add few-shot examples?
• How can you use prompting to reduce hallucinations?
• Your product team needs model output that software can read reliably. How do you design the prompt?
• Describe your process for improving a prompt that is giving mixed results.
• Explain temperature and top-p. How would you set them for different tasks?
• What are tokens and context windows, and why do they matter when you design prompts?
• What is chain-of-thought prompting, and when would you use it or avoid it?
• A prompt works on your demo examples but fails for some users. How do you find out why?