hard20 pts · first answer countsllm-evaluationrag-evalllm-as-judge

Relevance labels from an LLM

To evaluate retrieval cheaply, you have an LLM label 20,000 (query, chunk) pairs as relevant or not. What is the key step before trusting metrics built on these labels?

A.Ask the LLM to label each pair twice and keep the firstB.Use the labels directly since LLMs read faster than humansC.Compare a sample of LLM labels with human labels and measure agreementD.Only keep pairs the LLM marked relevant

Discussion 0

Join free to post in the discussion.

Relevance labels from an LLM

Discussion 0

Related questions