hard20 pts · first answer countsllm-as-judgeevaluation

Judge and candidate from one family

You compare outputs from model X and model Y using model X as the judge. X wins 70% of the time. What is the main concern?

A.None; judges are objectiveB.Self-preference bias: judges can favour outputs that resemble their own style, so check against human labels or use a judge from another familyC.The judge needs a higher temperatureD.Model Y outputs should be shown first

← Prompt cache hit rate near zero A suspicious score jump →