hard20 pts · first answer countsllm-evaluationmetrics
Auto-approving refunds safely
Refunds are auto-approved above a model score threshold, and the business needs at least 95% of auto-approvals to be correct. On labelled data: at 0.9, 400 approved and 388 correct; at 0.8, 600 approved and 564 correct; at 0.7, 750 approved and 690 correct. Which threshold should you choose?
Discussion 0
Join free to post in the discussion.
Related questions
- What exact match measuresPrompt Engineer
- Accuracy from four countsPrompt Engineer
- What a false positive isPrompt Engineer
- 99% accuracy that is uselessPrompt Engineer
- Code checks or an LLM judgePrompt Engineer
- Tracking broken JSONPrompt Engineer
- Grading with or without a referencePrompt Engineer
- Offline and online evaluationPrompt Engineer