medium10 pts · first answer countsllm-as-judgerubrics

Vague 1-10 judge scores

Your LLM judge rates answers "quality 1 to 10", and scores cluster at 7 or 8 regardless of real differences. How do you improve it?

A.Switch to a 1 to 100 scaleB.Ask the judge to be harsherC.Average the scores of three runsD.Replace the single score with specific pass/fail checks, such as "states the refund window correctly", each with a definition

Discussion 0

Join free to post in the discussion.

Vague 1-10 judge scores

Discussion 0

Related questions