easy5 pts · first answer countsllm-evaluationeval-design

When the answer key is wrong

While reviewing failures, you find that 6% of "failed" cases were actually correct; the golden labels were wrong. What should you do?

A.Keep the wrong labels so scores stay comparableB.Fix the labels and re-check others with a second reviewerC.Delete the whole eval set and start againD.Change the prompt until it matches the wrong labels

Discussion 0

Join free to post in the discussion.

When the answer key is wrong

Discussion 0

Related questions