hard20 pts · first answer countsevaluationobservability

Evals green, users unhappy

Your prompt scores 95% on the offline eval set, but user satisfaction keeps falling. What is the most likely gap and next step?

A.Users are using the product wronglyB.The eval set no longer reflects real traffic; sample and review production conversations, mine failures and feedback, and add them to the eval setC.The eval needs a higher passing thresholdD.Switch the judge model

← Long agent run fills context Take the hard mock interview →