Skip to content
RED SAGSkills

RAG interview questions and answers

RAG (retrieval-augmented generation) questions test how you ground an LLM in your own documents and measure whether the answers are right.

Below are 60 RAG questions with the correct answer and a short explanation, grouped from fresher to experienced level. Try each one before you open the answer.

Take the RAG MCQ testPractise 246+ RAG questions for points

What RAG interviews usually check

  • ✓Chunking and embeddings
  • ✓Vector search and hybrid search
  • ✓Reranking and context windows
  • ✓Evaluating retrieval and answers
  • ✓Hallucinations and citations

RAG interview questions for freshers easy

1. What text embeddings capture

What is the key property of a good text embedding model?

  1. A.Each dimension stores one dictionary word
  2. B.The vectors encode only the length of the text
  3. C.Texts with similar meaning map to nearby vectors
  4. D.Similar texts always get identical vectors
Show answer

Answer: C. Texts with similar meaning map to nearby vectors

Embeddings map text to dense vectors so that semantic similarity becomes geometric closeness, usually measured with cosine similarity. That is what makes semantic search, clustering and retrieval for RAG work. Dimensions are not individually interpretable words.

2. What RAG does

What does retrieval-augmented generation (RAG) do?

  1. A.Retrieves relevant documents for a query and adds them to the prompt so the model answers using them
  2. B.Retrains the model on every user question
  3. C.Randomly augments training data to reduce overfitting
  4. D.Generates images from retrieved text
Show answer

Answer: A. Retrieves relevant documents for a query and adds them to the prompt so the model answers using them

RAG searches a knowledge source (often with embeddings and a vector database) for passages related to the question, then places them in the prompt. The model grounds its answer in this context, which helps with private or recent information and allows citations.

3. What semantic search adds over keywords

A customer searches "how to get my money back" and the right help article is titled "Refund policy". Keyword search misses it. Why does embedding-based (semantic) search find it?

  1. A.It searches for every possible spelling of the query words
  2. B.It compares meaning, so different words with the same intent land close together
  3. C.It always returns the most popular article on the site
  4. D.It translates the query into SQL and runs it on the article table
Show answer

Answer: B. It compares meaning, so different words with the same intent land close together

Embedding models map text to vectors so that texts with similar meaning are near each other, even with no words in common. "Get my money back" and "refund" end up close. Keyword search needs overlapping terms, which is why many systems combine both.

4. What top-k means in retrieval

Your RAG retriever is set to top_k = 5. What does it return for each question?

  1. A.The first 5 chunks that were ever added to the index
  2. B.The 5 chunks whose vectors are closest to the query's
  3. C.5 random chunks so that answers have variety
  4. D.Chunks that contain at least 5 of the query's words
Show answer

Answer: B. The 5 chunks whose vectors are closest to the query's

The query is embedded and the index finds the k nearest stored vectors by the chosen similarity measure. Those chunks are passed to the LLM as context. k is a trade-off: too small can miss the answer, too large adds noise and cost.

5. Long chunks and embedding limits

Your embedding model accepts at most 512 tokens. You embed 2,000-token chunks without checking. What usually happens?

  1. A.The model averages all 2,000 tokens with no loss
  2. B.The vector becomes four times longer
  3. C.The embedding fails and returns an empty vector
  4. D.Text past 512 tokens is cut off and ignored
Show answer

Answer: D. Text past 512 tokens is cut off and ignored

Many embedding libraries silently truncate input to the model's limit. Facts in the dropped part can never be found by search, even though the full chunk text is stored. Size chunks to fit the embedding model, and log when truncation happens.

6. Sparse versus dense vectors

What is the difference between sparse vectors (as in BM25 or SPLADE) and dense embedding vectors?

  1. A.Sparse vectors are always more accurate than dense embedding vectors
  2. B.Dense vectors only work for English text, not other languages
  3. C.Sparse vectors are for images and dense vectors are for text
  4. D.Sparse: a slot per term, mostly zeros; dense: fewer slots, all used
Show answer

Answer: D. Sparse: a slot per term, mostly zeros; dense: fewer slots, all used

A sparse vector has a dimension for each vocabulary term and is non-zero only for terms present (or expanded), which suits inverted indexes and exact keyword matches. A dense vector has a few hundred to a few thousand dimensions, all used, and captures meaning. Hybrid search combines both.

7. When document and query vectors are made

In a typical RAG system, when are document embeddings and query embeddings computed?

  1. A.Both at indexing time, before any user asks anything
  2. B.Both at question time, for every document and the query
  3. C.Documents at indexing time; each query when the user asks it
  4. D.Documents are never embedded; only queries are
Show answer

Answer: C. Documents at indexing time; each query when the user asks it

Documents are chunked and embedded when they are ingested, and the vectors are stored. A query can only be embedded once it arrives, which adds a small latency at search time. Re-embedding documents is needed only when they change or you switch models.

8. How a recursive text splitter works

A "recursive character" text splitter is set with separators ["\n\n", "\n", ". ", " "]. How does it split a long document?

  1. A.It splits on every space first, then joins words back into paragraphs
  2. B.Paragraphs first; smaller separators only for pieces still too big
  3. C.It picks one of the separators at random for each new chunk
  4. D.It ignores separators and cuts every 500 characters
Show answer

Answer: B. Paragraphs first; smaller separators only for pieces still too big

The splitter tries the largest natural boundary first, here blank lines between paragraphs. Any piece still over the size limit is split again on single new lines, then sentences, then words. This keeps chunks aligned with the text's own structure where possible.

9. Scanned PDFs return no text

Ingestion finds zero text in a batch of PDFs, though they clearly show contract pages when opened. What is the likely reason?

  1. A.They are scanned images with no text layer and need OCR
  2. B.The PDFs are too large for the vector database to store
  3. C.The embedding model cannot read legal language
  4. D.PDF files can only be indexed by keyword search
Show answer

Answer: A. They are scanned images with no text layer and need OCR

A scanned PDF stores pictures of pages, not characters. Text extraction finds nothing, so nothing gets embedded. Run OCR (or a vision model) to produce text, and check OCR quality on Arabic or mixed-script pages before trusting it.

10. Rewriting the user query

Users type messages like "hi, quick q, the thing about leaving early on fridays??". Why do some RAG systems ask an LLM to rewrite the query before searching?

  1. A.To translate the user's question into SQL for the vector database
  2. B.To shorten the user's question so it costs fewer tokens to answer
  3. C.To turn chatty text into a clear search query such as "Friday early leave policy"
  4. D.To check the user's spelling before showing the answer
Show answer

Answer: C. To turn chatty text into a clear search query such as "Friday early leave policy"

Greetings, filler and vague words add noise to the query embedding and keyword match. A short rewrite step extracts the real information need. Keep the original question for the final answer prompt, and monitor rewrites for meaning drift.

11. Restricting search to the right year

A user asks, "What was the travel allowance in the 2024 policy?" The index has 2022 to 2026 versions of the policy, all very similar. What helps most?

  1. A.Retrieve more chunks and hope the 2024 one is included
  2. B.Delete all versions except the newest one
  3. C.Ask the LLM to guess which version is from 2024
  4. D.Filter the search to chunks whose metadata year is 2024
Show answer

Answer: D. Filter the search to chunks whose metadata year is 2024

Similar versions produce nearly identical vectors, so similarity alone cannot pick the right year. A metadata filter on year (or effective date) narrows the search to the correct version. This requires capturing that metadata reliably at ingestion.

12. What retrieval hit rate measures

A RAG dashboard reports "hit rate@5 = 82%". What does that mean?

  1. A.82% of questions had a relevant chunk in the top 5
  2. B.82% of all the retrieved chunks were relevant
  3. C.82% of users clicked at least one citation link
  4. D.The answer cache handled 82% of the questions
Show answer

Answer: A. 82% of questions had a relevant chunk in the top 5

Hit rate (or success@k) counts a question as a hit if any relevant chunk appears in the top k. It is simple and good for questions with one key passage. It does not reward ranking the chunk higher, which MRR and nDCG do.

13. What a golden eval set contains

Before changing your chunking, you want a "golden" evaluation set for the RAG bot. What should it contain?

  1. A.Realistic questions with correct answers and their sources
  2. B.Only the questions that users rated with a thumbs-up
  3. C.A full copy of every document stored in the index
  4. D.A list of the embedding model's settings and parameters
Show answer

Answer: A. Realistic questions with correct answers and their sources

A golden set pairs representative questions with correct answers and the chunks or documents that contain them. You can then score retrieval (did we find the source?) and generation (is the answer right?) before and after any change. Include hard and unanswerable questions too.

14. What faithfulness measures

In RAG evaluation, what does a "faithfulness" (groundedness) score measure?

  1. A.Whether the user trusted the answer enough to stop asking
  2. B.Whether the retriever returned chunks in date order
  3. C.Whether the claims in the answer are supported by the retrieved context
  4. D.Whether the answer used the same words as the question
Show answer

Answer: C. Whether the claims in the answer are supported by the retrieved context

Faithfulness checks that each statement in the answer can be backed by the retrieved text, so the model is not adding unsupported facts. It is usually scored by breaking the answer into claims and checking each against the context. A faithful answer can still be wrong if the retrieved context was wrong.

15. What answer relevance measures

An answer is fully supported by the context but talks about sick leave when the user asked about annual leave. Which metric should flag this?

  1. A.Faithfulness
  2. B.Index recall
  3. C.Chunk overlap
  4. D.Answer relevance
Show answer

Answer: D. Answer relevance

Answer relevance checks whether the response actually addresses the question asked. Faithfulness can be perfect here because every claim is in the context, yet the answer is still useless. Tracking both catches different failure types.

16. What LLM-as-a-judge means

Your team uses "LLM-as-a-judge" to evaluate the RAG bot. What does that involve?

  1. A.Letting the chat model decide which users may use the bot
  2. B.Asking the LLM to choose which documents to delete
  3. C.Using an LLM with a clear rubric to grade answers at scale
  4. D.Training the chat model with legal court cases
Show answer

Answer: C. Using an LLM with a clear rubric to grade answers at scale

A judge model reads the question, the context, the answer and sometimes a reference, then scores it against criteria like correctness or faithfulness. It is much cheaper than human review for thousands of cases. Check its scores against human labels before trusting it.

17. Questions about last week

Users ask your assistant about news from last week, and it gives outdated or invented answers. What is the right fix?

  1. A.Tell the model in the prompt that it knows about all recent news
  2. B.Retrieve current information, for example with search, and put it in the prompt
  3. C.Raise the temperature so it guesses more freely
  4. D.Repeat the question twice in the prompt
Show answer

Answer: B. Retrieve current information, for example with search, and put it in the prompt

A model only knows what was in its training data up to its knowledge cutoff. For recent facts, you must supply the information at request time, through retrieval, search tools or pasted documents. Telling it that it knows does not add knowledge.

18. What RAG means

What is retrieval-augmented generation (RAG)?

  1. A.Training a new model from scratch on company data
  2. B.Searching a knowledge source for relevant text and adding it to the prompt so the model answers from it
  3. C.Generating random answers and keeping the best one
  4. D.Compressing the model so it runs on a phone
Show answer

Answer: B. Searching a knowledge source for relevant text and adding it to the prompt so the model answers from it

RAG first retrieves passages related to the question, often from a vector database or search index, and then puts them in the prompt. The model answers using that context. It lets you use private or fresh information without retraining the model.

19. What embeddings are for

In a RAG system, why are documents converted into embeddings?

  1. A.To make the documents smaller for email
  2. B.To translate them into English
  3. C.To turn text into number vectors where similar meanings are close, so you can search by meaning
  4. D.To remove personal data from them
Show answer

Answer: C. To turn text into number vectors where similar meanings are close, so you can search by meaning

An embedding model maps text to a vector. Passages about the same idea land near each other even if they use different words. At query time you embed the question and find the closest passages, which is called semantic or vector search.

20. Asking about last week's news

A user asks your assistant about RBI policy changes announced last week. The model gives outdated information. What is the root cause?

  1. A.The temperature is too low for news questions
  2. B.The model only knows data up to its training cutoff
  3. C.The system prompt is longer than the question
  4. D.RBI content is blocked by every model provider
Show answer

Answer: B. The model only knows data up to its training cutoff

A model's built-in knowledge stops at its training cutoff, so recent events are unknown or mixed up with older ones. Giving it fresh information through search or retrieval, with dates, is the fix.

RAG intermediate interview questions medium

21. Embeddings miss exact product codes

Users search for exact codes like "INV-20931" or "error E1047", and pure embedding search often returns the wrong documents. What helps most?

  1. A.Hybrid search: combine keyword search (such as BM25) with vector search and merge the results
  2. B.Use a bigger embedding model only
  3. C.Remove all numbers from documents before indexing
  4. D.Lower the number of results returned to one
Show answer

Answer: A. Hybrid search: combine keyword search (such as BM25) with vector search and merge the results

Embeddings capture meaning but are weak at exact matching of rare strings like IDs and codes. Keyword search is very good at that. Hybrid search runs both and fuses scores (for example with reciprocal rank fusion), giving the best of both.

22. Adding a reranker

In a RAG pipeline, you retrieve 50 candidate chunks with vector search and then run a cross-encoder reranker to keep the top 5. Why?

  1. A.The reranker creates new documents that are not in the index
  2. B.The reranker reduces the size of the vector index
  3. C.A cross-encoder reads query and chunk together, giving more accurate relevance scores than embedding similarity, but it is too slow to run on the whole corpus
  4. D.The reranker translates documents into the user language
Show answer

Answer: C. A cross-encoder reads query and chunk together, giving more accurate relevance scores than embedding similarity, but it is too slow to run on the whole corpus

Bi-encoder embeddings are fast because document vectors are computed in advance, but they compare query and document separately. A cross-encoder scores each pair jointly and is more accurate but expensive, so it is used on a small candidate set. This two-stage design improves precision of the final context.

23. Computing recall at 5

A query has 3 relevant chunks in the index. Your retriever returns 5 chunks, and 2 of them are relevant. What are recall@5 and precision@5?

  1. A.Recall@5 = 0.40, precision@5 = 0.67
  2. B.Recall@5 = 0.67, precision@5 = 0.67
  3. C.Recall@5 = 0.67, precision@5 = 0.40
  4. D.Recall@5 = 1.00, precision@5 = 0.40
Show answer

Answer: C. Recall@5 = 0.67, precision@5 = 0.40

Recall@5 = relevant found / all relevant = 2/3 ≈ 0.67. Precision@5 = relevant found / results returned = 2/5 = 0.40. For RAG, recall matters most because the model cannot use a chunk that was never retrieved.

24. Computing MRR

For three test queries, the first relevant result appears at rank 1, at rank 3, and not at all in the top 10. What is the MRR@10?

  1. A.0.44
  2. B.0.67
  3. C.0.33
  4. D.0.58
Show answer

Answer: A. 0.44

Reciprocal ranks are 1/1 = 1, 1/3 ≈ 0.333, and 0 for the miss. The mean is (1 + 0.333 + 0) / 3 ≈ 0.44. MRR rewards putting the first relevant result near the top. A common mistake is to average only over queries with a hit, which would give 0.67.

25. Reciprocal rank fusion scores

You fuse BM25 and vector results with reciprocal rank fusion, score = sum of 1 / (60 + rank). Doc A is rank 1 in BM25 only. Doc B is rank 5 in both lists. Which ranks higher?

  1. A.Doc B, about 0.031 vs 0.016
  2. B.Doc A, about 0.016 vs 0.008
  3. C.Doc A, because rank 1 always wins
  4. D.They tie, because both have a total rank of about 5
Show answer

Answer: A. Doc B, about 0.031 vs 0.016

Doc A scores 1/61 ≈ 0.0164. Doc B scores 1/65 + 1/65 = 2/65 ≈ 0.0308. RRF rewards documents that both retrievers agree on, and the constant 60 keeps any single top rank from dominating.

26. What SPLADE adds to keyword search

SPLADE produces sparse vectors over the vocabulary. What advantage does it have over plain BM25?

  1. A.It removes the need for any index structure at all
  2. B.It stores dense 768-dimension vectors for every word
  3. C.It only matches exact spellings, never related words
  4. D.It learns term weights and adds related terms not in the text
Show answer

Answer: D. It learns term weights and adds related terms not in the text

SPLADE uses a language model to assign weights to vocabulary terms, including expansion terms, so a passage about "refunds" can also score on "money back". The output is still sparse, so it runs on inverted indexes like BM25. It keeps exact-term strengths while adding some semantic matching.

27. Shards versus replicas in a vector DB

Your vector index fits on one node, but peak traffic causes high query latency. Should you add shards or replicas?

  1. A.Replicas, so more nodes can answer queries in parallel
  2. B.Shards, because they make each query use fewer nodes
  3. C.Neither, because vector databases cannot scale traffic
  4. D.Shards, because replicas double the embedding cost
Show answer

Answer: A. Replicas, so more nodes can answer queries in parallel

Sharding splits data across nodes and is mainly for data that does not fit on one machine; each query fans out to every shard. Replicas copy the same data so more queries can be served at once. For a traffic problem with data that fits, replicas are the direct fix.

28. Setting BM25 b to zero

You set BM25's b parameter to 0. What changes?

  1. A.Term frequency is ignored, so each word counts only once
  2. B.Length is ignored, so long documents tend to score higher
  3. C.Rare words lose their extra IDF weight in the score
  4. D.All documents get exactly the same score of zero
Show answer

Answer: B. Length is ignored, so long documents tend to score higher

b (usually around 0.75) scales term frequency by document length relative to the average. With b = 0 there is no length normalisation, so long documents that mention a term more often win. With b = 1, normalisation is full. Tune it when your corpus mixes very short and very long texts.

29. Routing questions to different sources

Your assistant covers HR policies, IT runbooks and finance SOPs in separate indexes. Searching all three for every question returns mixed, off-topic chunks. What helps?

  1. A.A router that picks the right index for each question
  2. B.Merging all three into one index without any metadata
  3. C.Always searching only the largest of the three indexes
  4. D.Raising top_k to 50 for every index on each question
Show answer

Answer: A. A router that picks the right index for each question

A lightweight classifier or LLM call can decide that "VPN not connecting" belongs to IT and "maternity leave" to HR. Searching only the relevant source improves precision and saves work. Allow multiple routes for ambiguous questions and log routing decisions for evaluation.

30. Reranking time for 100 candidates

Your cross-encoder reranker on one GPU scores a batch of 25 query-chunk pairs in 40 ms, and batches run one after another. How long does reranking 100 candidates take?

  1. A.About 40 ms
  2. B.About 400 ms
  3. C.About 160 ms
  4. D.About 1,000 ms
Show answer

Answer: C. About 160 ms

100 candidates ÷ 25 per batch = 4 batches, and 4 × 40 ms = 160 ms. If that is too slow for your latency budget, rerank fewer candidates, use a smaller reranker, run batches on more GPUs, or truncate long chunks.

31. Using an LLM as a reranker

A team replaces its cross-encoder with an LLM that reads 20 candidate chunks and returns them in order of relevance. What is the main trade-off?

  1. A.Always faster and cheaper, with no change in quality
  2. B.Often better judgement on hard queries, but more latency and cost
  3. C.It removes the need for any first-stage retrieval
  4. D.It can only rerank results in English
Show answer

Answer: B. Often better judgement on hard queries, but more latency and cost

LLM rerankers can reason about subtle relevance and follow instructions such as "prefer the 2025 policy". But they are slower and pricier than small cross-encoders, can be sensitive to the order of candidates in the prompt, and may drop or invent IDs. Validate output and measure gains on your eval set.

32. What late chunking does

"Late chunking" uses a long-context embedding model in a particular way. What is it?

  1. A.Encode the whole document once, then pool token vectors per chunk
  2. B.Chunk the document only after the user asks a question
  3. C.Embed each chunk separately and average all of them
  4. D.Wait until the index is full before chunking
Show answer

Answer: A. Encode the whole document once, then pool token vectors per chunk

Normal chunking embeds each chunk in isolation, so a chunk saying "it" loses what "it" refers to. Late chunking runs the full document (or a large window) through the model first, so every token vector carries document context, then averages token vectors within each chunk's span. Chunk vectors become more context-aware.

33. Letting the model search again

In "agentic RAG", the LLM can call the search tool several times. What does this add over a single fixed retrieval step?

  1. A.It removes the need for an index because the model knows everything
  2. B.It guarantees the answer is correct after three searches
  3. C.It makes each question cost exactly one search call
  4. D.It can refine the query and search again when results are not enough
Show answer

Answer: D. It can refine the query and search again when results are not enough

With a single step, a poor first search means a poor answer. An agent can read the results, notice gaps, and issue follow-up searches with new terms or filters. The cost is more latency and tokens, so cap the number of steps and log each tool call.

34. Offline versus online RAG evaluation

How do offline and online evaluation of a RAG system differ?

  1. A.Offline runs on a laptop; online runs in the cloud with the same test set
  2. B.Offline measures cost only; online measures accuracy only
  3. C.Offline: a fixed labelled set; online: live traffic and outcomes
  4. D.Offline is for retrieval; online is only for the language model
Show answer

Answer: C. Offline: a fixed labelled set; online: live traffic and outcomes

Offline evaluation is repeatable and safe: you score candidates on the same golden set before shipping. Online evaluation, such as A/B tests, feedback, escalation or resolution rates, shows real impact but is noisier and slower. Use offline checks to gate releases and online metrics to confirm them.

35. Arabic question, English sources

Your sources are English, and an Emirati user asks in Arabic. Retrieval works, but the bot replies in English. What is the best fix?

  1. A.Translate the whole corpus into Arabic before every new query
  2. B.Tell the model to reply in the user's language, citing sources
  3. C.Refuse any questions that are not written in English
  4. D.Switch to an Arabic-only embedding model for all queries
Show answer

Answer: B. Tell the model to reply in the user's language, citing sources

The model tends to mirror the language of the context it reads. A clear instruction to reply in the question's language fixes this, and citations still point to the English source. Check terminology for legal or policy terms, and consider keeping key quotes in the original language.

36. Building an eval set from logs

You want your eval set to reflect real usage. What is a sound way to build it from production logs?

  1. A.Sample across topics and difficulty, redact, then label
  2. B.Take the 100 most common queries, since rare ones do not matter
  3. C.Use only queries that received a thumbs-up
  4. D.Copy every log line into the eval set without review
Show answer

Answer: A. Sample across topics and difficulty, redact, then label

Stratified sampling keeps both frequent and long-tail topics, including hard and unanswerable questions. Redact personal data, then have experts write or check reference answers and sources. Only-positive or only-common samples hide the failures you most need to see.

37. Switching embedding models in RAG

You replace the embedding model used by a RAG system with a better one. What must happen to the existing vector index?

  1. A.Nothing; vectors from different models are compatible
  2. B.Only new documents need the new model
  3. C.Normalise the old vectors and keep them
  4. D.Re-embed the whole corpus with the new model and build a new index
Show answer

Answer: D. Re-embed the whole corpus with the new model and build a new index

Each embedding model has its own vector space, so distances between old and new vectors are meaningless, even when the dimensions match. Query and document vectors must come from the same model. Build the new index alongside the old one, evaluate retrieval and then switch over.

38. Checkable citations

You want every claim in a RAG answer to be traceable. What prompt design makes this easiest to check?

  1. A.Ask the model to "be accurate"
  2. B.Ask it to list its training data
  3. C.Give each retrieved chunk an ID and require the answer to cite the IDs it used after each claim
  4. D.Put all chunks into one unlabeled block of text
Show answer

Answer: C. Give each retrieved chunk an ID and require the answer to cite the IDs it used after each claim

Labeled chunks like [doc-3] let the model cite exactly where a claim came from. Your code can confirm each ID exists and show sources to users, and evaluators can check whether the cited chunk supports the claim. Without IDs, citations are vague and hard to verify.

39. Follow-ups that retrieve nothing

In a RAG chatbot, a user asks "What is the fee for the gold card?" then "And for platinum?". The second search returns junk. What fixes this?

  1. A.Search with the system prompt instead of the user message
  2. B.Increase the number of retrieved chunks from 5 to 50
  3. C.Ask the user to repeat the full question every time
  4. D.Rewrite the follow-up into a standalone query using the chat history
Show answer

Answer: D. Rewrite the follow-up into a standalone query using the chat history

"And for platinum?" has no keywords about fees on its own. A query-rewriting step turns it into "What is the fee for the platinum card?" before retrieval, which matches the right documents.

40. Chatty questions to a keyword index

Your search backend is keyword-based (BM25). Users write long messages like "hi, so I was wondering whether I can maybe get GST invoices for last year's orders?". What prompt step helps retrieval?

  1. A.Send the message to search exactly as the user typed it
  2. B.Translate the message into French first
  3. C.Rewrite the message into key search terms first
  4. D.Ask the user to shorten their messages
Show answer

Answer: C. Rewrite the message into key search terms first

Keyword search works best with focused terms like "GST invoice previous year orders". An LLM rewrite step removes filler and adds key terms, which improves recall and precision of the search.

RAG interview questions for experienced candidates hard

41. Stuffing 100 chunks into the prompt

To "improve recall", a team puts 100 retrieved chunks into a long-context prompt. Answer quality drops compared to using the top 8. What is the most likely reason?

  1. A.Relevant facts get buried among many distractors; models often use information in the middle of long contexts less reliably
  2. B.Long context models cannot read more than 8 chunks
  3. C.The vector database returns random results after 8
  4. D.Temperature automatically increases with prompt length
Show answer

Answer: A. Relevant facts get buried among many distractors; models often use information in the middle of long contexts less reliably

More context is not always better. Irrelevant chunks add noise, raise cost and latency, and models can pay less attention to information buried in the middle of a long prompt. Better retrieval, reranking and putting the most relevant content in prominent positions usually work better.

42. Budgeting the context window

Your model has a 128,000-token context window. The system prompt uses 2,000 tokens, tool definitions 6,000, chat history 30,000, and you reserve 4,000 for the answer. How many 800-token retrieved chunks fit at most?

  1. A.107
  2. B.112
  3. C.160
  4. D.102
Show answer

Answer: A. 107

Free space = 128,000 − 2,000 − 6,000 − 30,000 − 4,000 = 86,000 tokens. 86,000 / 800 = 107.5, so 107 whole chunks fit. Forgetting the output reserve gives 112, which would leave too little room for the answer. In practice you would use far fewer chunks for quality reasons.

43. IVF index after a new language

An IVF index was trained on English documents. You then add 3 lakh Arabic documents. Latency spikes for Arabic queries and recall drops. What is likely happening?

  1. A.IVF indexes can only store one language at a time
  2. B.Arabic vectors are larger, so the index runs out of disk
  3. C.The new vectors crowd into a few old clusters; retrain the centroids
  4. D.The Arabic documents are being ignored by the index
Show answer

Answer: C. The new vectors crowd into a few old clusters; retrain the centroids

IVF centroids come from k-means on the data present at training time. If new data lies in a different region, it piles into a few lists, making them huge and slow, while the clustering no longer separates it well. Re-train the IVF on a representative sample and rebuild.

44. Inner product with unnormalised vectors

Your index uses inner product (dot product), and the embeddings are not normalised. A few long chunks show up for almost every query. Why?

  1. A.Long chunks are always more relevant than short ones
  2. B.Vectors with large norms get high dot products with many queries
  3. C.Inner product ignores vector length completely
  4. D.The index sorts results by chunk length
Show answer

Answer: B. Vectors with large norms get high dot products with many queries

Dot product = ‖a‖ × ‖b‖ × cos. If some vectors have larger norms, they score high regardless of direction. Normalise vectors to unit length (making dot product equal cosine) unless the model's docs say the norm carries useful meaning.

45. One vector for a 40-page PDF

To save cost, a team embeds each 40-page PDF as a single vector (using a long-context embedding model). Specific questions now retrieve poorly. Why?

  1. A.Long-context embedding models always return random vectors
  2. B.The vector database cannot store vectors for long files
  3. C.One vector blends many topics, so no single detail stands out
  4. D.Specific questions need keyword search and never work with vectors
Show answer

Answer: C. One vector blends many topics, so no single detail stands out

An embedding is a summary; for a long document it reflects the overall theme and dilutes any one clause. A question about a single penalty rule then matches poorly. Chunk the document, or keep a document-level vector for routing plus chunk vectors for detail.

46. Answer split across two chunks

A rule reads "Claims must be filed within 30 days" at the end of chunk 41, and "unless the employee was abroad, in which case 60 days" starts chunk 42. Only chunk 41 is retrieved, so the answer is incomplete. What helps without re-chunking?

  1. A.Raise the LLM temperature so it guesses the exceptions
  2. B.Lower top_k so that only the strongest chunk is used
  3. C.Also fetch the neighbouring chunks (n − 1 and n + 1) of each hit
  4. D.Delete chunk 42 because it starts mid-sentence
Show answer

Answer: C. Also fetch the neighbouring chunks (n − 1 and n + 1) of each hit

Storing each chunk's position lets you expand a hit with the chunks next to it, a pattern also called sentence-window or context expansion. Exceptions and conditions often sit just past a boundary. Watch the extra tokens, and prefer structure-aware boundaries in future ingestion.

47. Questions about the whole corpus

Leadership asks the RAG bot, "What are the main complaint themes across all 40,000 support tickets this year?" Answers are shallow and miss big themes. Why, and what fits better?

  1. A.Top-k sees few tickets; use map-reduce or GraphRAG summaries
  2. B.The embedding model is too small; switch to a larger dimension
  3. C.The bot needs a higher temperature to think more broadly
  4. D.Tickets must be retrieved in date order to see themes
Show answer

Answer: A. Top-k sees few tickets; use map-reduce or GraphRAG summaries

Standard RAG retrieves a handful of chunks similar to the question, but a "global" question needs evidence from the whole corpus. Approaches like map-reduce summarisation, clustering, or GraphRAG (entity graphs with pre-computed community summaries) aggregate across everything. For counts, structured analytics may be better still.

48. Garbled chunks from two-column PDFs

Chunks from research reports read like "The tariff on imports rose 4% Section 3 covers the Dubai free-zone rules sharply in Q2". What is the cause and fix?

  1. A.The embedding model mixes sentences; switch embedding models
  2. B.The extractor reads across both columns; use a layout-aware parser
  3. C.Overlap is too large; set chunk overlap to zero
  4. D.The reports are in two languages; translate them first
Show answer

Answer: B. The extractor reads across both columns; use a layout-aware parser

Basic PDF extraction can read lines left to right across the page, merging two columns into nonsense. Layout-aware tools detect columns, headers and footers and read in the correct order. Spot-check parsed text from each document type; bad parsing silently ruins retrieval.

49. Re-runs create duplicate chunks

Each time the ingestion job is re-run after a failure, the index gains another copy of many chunks. Chunk IDs are random UUIDs. What is the fix?

  1. A.Run the ingestion job only once a year to avoid duplicates
  2. B.Use deterministic IDs (document ID plus index) and upsert
  3. C.Delete the entire index and rebuild it before every run
  4. D.Ask the LLM to ignore repeated chunks in the context
Show answer

Answer: B. Use deterministic IDs (document ID plus index) and upsert

With random IDs, the store cannot tell that a chunk already exists, so every retry inserts it again. A deterministic ID derived from the source and position (or content hash) turns inserts into idempotent upserts. Re-runs then overwrite instead of duplicating.

50. RAG gets totals wrong

Finance users ask, "What was total GST paid across all branches in Q3?" The RAG bot retrieves 5 invoice chunks and gives a wrong total. What is the right design?

  1. A.Route them to a SQL tool over the structured data
  2. B.Retrieve 500 invoice chunks so the model can add them up
  3. C.Ask the model to estimate based on the 5 invoices it sees
  4. D.Embed the invoices with a larger model so totals are captured
Show answer

Answer: A. Route them to a SQL tool over the structured data

Aggregations need every relevant record and exact arithmetic, but top-k retrieval returns a small sample and LLMs add numbers unreliably. Detect analytic questions and send them to a text-to-SQL or analytics tool over the source tables. Use RAG for policy and explanation questions.

51. Generating questions for each chunk

Instead of rewriting queries at search time, a team uses an LLM at ingestion to write 3 likely questions per chunk and indexes them with the chunk. What is the main trade-off?

  1. A.No ingestion cost, but every query now needs three LLM calls
  2. B.Higher ingestion cost and index size, but no extra latency at query time
  3. C.It lowers recall, since questions never match user questions
  4. D.It removes the need to store the original chunk text
Show answer

Answer: B. Higher ingestion cost and index size, but no extra latency at query time

This index-time expansion (like doc2query) makes chunks easier to match with question-style queries. The LLM work happens once per chunk, so search stays fast. You pay for generation over the whole corpus and store more vectors or text, and bad generated questions can add noise.

52. Suspiciously high eval scores

After adding few-shot examples to the RAG prompt, your eval score jumped from 71% to 93%. Later you find the examples were copied from the eval set. What is the problem?

  1. A.Test leakage: the eval now measures memorised examples, not real ability
  2. B.Few-shot examples always lower accuracy, so the score must be wrong
  3. C.The eval set is too small to contain any examples
  4. D.The judge model was updated at the same time
Show answer

Answer: A. Test leakage: the eval now measures memorised examples, not real ability

If test questions or their answers appear in the prompt, the model can copy them, inflating scores without improving real performance. Keep prompt examples, tuning data and eval sets separate, and hold out a set that nobody tunes against.

53. False facts planted in a wiki

Your RAG indexes an internal wiki that any employee can edit. Someone adds a page with a wrong refund rule, and the bot starts quoting it to customers. What reduces this risk?

  1. A.Index more wiki pages so the wrong one is outnumbered
  2. B.Rank sources by trust and review edits to open sources
  3. C.Raise the temperature so the bot varies its answers
  4. D.Remove citations so customers cannot see the source
Show answer

Answer: B. Rank sources by trust and review edits to open sources

Open-edit sources let anyone insert content the bot will treat as truth. Tag sources by authority, filter or rank approved policy documents above wiki pages for customer answers, and route changes to sensitive topics through review. Monitoring which sources answers cite also catches surprises.

54. Better offline, flat online

A new reranker raised offline nDCG@5 from 0.71 to 0.80, but after an A/B test the online resolution rate did not change. What is the most useful next step?

  1. A.Conclude that rerankers never help and remove it
  2. B.Ship it anyway and stop measuring online metrics
  3. C.Check the eval set matches live traffic and find real failures
  4. D.Lower the nDCG cut-off from 5 to 1 so the gain looks bigger
Show answer

Answer: C. Check the eval set matches live traffic and find real failures

Offline gains only help if the eval reflects real queries and retrieval was the bottleneck. Compare the eval set with live query types, and sample failed online chats to see whether failures come from missing content, generation or UX. Then target the real bottleneck.

55. Data that must stay in the country

A UAE government client requires that its documents never leave in-country infrastructure. Your pipeline calls a foreign-hosted embedding API at ingestion and at query time. What must change?

  1. A.Nothing, because embeddings are just numbers
  2. B.Every model that sees the text must run in-country
  3. C.Only the vector database needs to be moved into the country
  4. D.Encrypt the vectors after they come back from the API
Show answer

Answer: B. Every model that sees the text must run in-country

Calling an embedding or LLM API sends the raw document and query text to that provider's servers. For residency requirements, every step that sees the text, including ingestion, query embedding, reranking and generation, must run in approved locations. Check the provider's regions or self-host models.

56. Retrieval got worse as the index grew

An HNSW vector index grew from 1 million to 50 million chunks with the same settings. Answer quality dropped, and recall@10 against exact search fell from 0.97 to 0.85. What should you tune first?

  1. A.Lower the embedding dimension to 2
  2. B.Raise the search-time candidate list (ef_search), and if needed rebuild with a higher M, measuring recall and latency
  3. C.Switch from cosine to random ranking
  4. D.Shrink the number of chunks returned to 1
Show answer

Answer: B. Raise the search-time candidate list (ef_search), and if needed rebuild with a higher M, measuring recall and latency

Approximate search trades recall for speed, and settings tuned for a small index may miss neighbours in a larger graph. ef_search controls how many candidates are explored per query, so raising it improves recall at some latency cost. Measuring recall against exact search on a sample keeps this tuning honest.

57. New embedding model, old index

You upgrade the query embedding model in a RAG system but keep the existing vector index. Retrieval quality collapses. Why?

  1. A.Vector indexes expire after an upgrade
  2. B.Old and new embeddings live in different vector spaces
  3. C.The new model has too many dimensions to search
  4. D.Query caching returns the old results
Show answer

Answer: B. Old and new embeddings live in different vector spaces

Vectors from two different embedding models are not comparable even with the same dimension. Documents must be re-embedded with the new model, ideally into a new index, with retrieval evals before switching traffic.

58. Real sources, wrong claims

Your RAG answers always cite real chunk IDs, but reviewers find some cited chunks do not actually support the claim. How should you catch this at scale?

  1. A.Run an automated check, such as a judge or entailment model, that tests whether each cited chunk supports its claim, and track that rate as a metric
  2. B.Trust the citations, since the IDs are real
  3. C.Remove citations from the UI
  4. D.Ask the model to cite more chunks per claim
Show answer

Answer: A. Run an automated check, such as a judge or entailment model, that tests whether each cited chunk supports its claim, and track that rate as a metric

A valid ID only proves the chunk exists, not that it says what the answer claims. Checking claim and source pairs measures faithfulness, often called groundedness. Flag or block unsupported claims and review samples by hand to calibrate the checker.

59. An expert persona that still errs

Adding "You are a world-class UAE tax lawyer" made answers sound more authoritative, but factual errors about VAT rules stayed the same. What is the best explanation and fix?

  1. A.The persona needs more adjectives, such as "senior" and "award-winning"
  2. B.Roles only work for programming tasks
  3. C.The model must be told it is in Abu Dhabi
  4. D.A role shapes tone, not facts; supply the current VAT rules as context
Show answer

Answer: D. A role shapes tone, not facts; supply the current VAT rules as context

A role prompt can change vocabulary, depth and style, but it cannot add knowledge the model lacks or update rules that changed after training. Grounding the answer in up-to-date regulation text (retrieval) is what fixes factual errors.

60. Six levels of categories on mobile

An industrial parts app has six category levels. On mobile, users get lost tapping through. Which approach helps most?

  1. A.Show all six levels together on one screen
  2. B.Remove four of the six levels at random
  3. C.Use a swipeable carousel at each level
  4. D.Strong search and filters, plus clear path and back
Show answer

Answer: D. Strong search and filters, plus clear path and back

Deep trees are hard on small screens. Search with filters lets experts jump straight to parts, while drill-down with a visible path and easy back supports browsing. Flattening the tree where possible also helps.

Ready to test yourself?

Take a 15-question RAG MCQ test with a score at the end, or practise the full RAG question bank and earn points on the leaderboard.

Start the RAG testTake a mock interview