hard20 pts · first answer countsllm-evaluationstatistics

Comparing two prompts on the same cases

Prompts A and B run on the same 500 cases. Both right: 380. Both wrong: 60. Only A right: 40. Only B right: 20. A McNemar test uses which numbers, and what does it suggest?

A.The 40 and 20; A is likely truly better (χ² ≈ 6.7)B.The 380 and 60; the prompts are identicalC.All 500; B is better because it has fewer winsD.Only the 380; neither prompt can be compared

Discussion 0

Join free to post in the discussion.

Comparing two prompts on the same cases

Discussion 0

Related questions