Prompt Engineer
Evaluation questions
19 questions on Evaluation for prompt engineers, easiest first.
Easy 2
Medium 4
Hard 13
- Perfect on 20 exampleshard
- Can we trust the LLM judgehard
- Switching to a newer modelhard
- Judge and candidate from one familyhard
- A suspicious score jumphard
- Is a 2-point gain real?hard
- Where is the RAG bug?hard
- Real sources, wrong claimshard
- Longer answers keep winninghard
- Tiny wording, big swinghard
- Errors flowing down a chainhard
- Scores change between runshard
- Evals green, users unhappyhard