hard20 pts · first answer countsdistillation

Why scale distillation loss by T squared

In knowledge distillation with softened logits at temperature T, why is the soft-target loss usually multiplied by T^2?

A.It turns the logits into valid probabilitiesB.Soft-target gradients shrink by about 1/T^2, so this restores scaleC.It makes the student network largerD.It stops the teacher from being updated

Discussion 0

Join free to post in the discussion.

Why scale distillation loss by T squared

Discussion 0

Related questions