hard20 pts · first answer countsfine-tuningevaluation

Fine-tuned model scores too well

Your fine-tuned support model scores 97% on the eval set but users report many failures. You find the training data was built from the same ticket export as the eval set. What is the most likely problem?

A.The learning rate was too low, so the model never fully convergedB.The LoRA rank was too high, so the adapter ignored the base modelC.Near-duplicates leaked across train and eval, inflating the scoreD.The eval set was run at temperature 0 instead of production settings

← Adding BM25 and cosine scores Field order in an output schema →