hard20 pts · first answer countsrerankingrag

How many results to rerank

You add a cross-encoder reranker after vector search. Why do teams rerank the top 50 to 100 candidates instead of all 100,000 documents?

A.Cross-encoder APIs accept at most 50 documents in a single requestB.Reranking more documents always lowers the final answer accuracyC.Results from vector search cannot be passed to a reranker directlyD.It runs a full model pass per query-document pair, so cost grows per candidate

← Serving a mixture-of-experts model Adding BM25 and cosine scores →