hard20 pts · first answer countsllm-evaluationmetricscode-eval
Computing pass@2
For one coding task, you sample 10 solutions and 3 pass the tests. Using the unbiased estimator pass@k = 1 − C(n−c, k) ÷ C(n, k), what is pass@2?
Discussion 0
Join free to post in the discussion.
Related questions
- What exact match measuresPrompt Engineer
- Accuracy from four countsPrompt Engineer
- What a false positive isPrompt Engineer
- 99% accuracy that is uselessPrompt Engineer
- Code checks or an LLM judgePrompt Engineer
- Tracking broken JSONPrompt Engineer
- Grading with or without a referencePrompt Engineer
- Offline and online evaluationPrompt Engineer