hard20 pts · first answer countsevaluationprompt-robustness

Tiny wording, big swing

Changing "Answer:" to "Response:" moves accuracy by 6 points on your eval. What does this tell you about how to evaluate prompts?

A.The word "Answer" is always betterB.Models are sensitive to small format changes, so test several paraphrases and formats and prefer prompts that score well robustlyC.The eval set must be brokenD.Formatting never matters once the model is large enough

← Longer answers keep winning Errors flowing down a chain →