medium10 pts · first answer countsfine-tuningsft

Model never stops after SFT

You set pad_token = eos_token and mask all pad positions with -100. After SFT, the model never stops generating. Why?

A.Padding makes the context window too shortB.EOS tokens are not allowed in training dataC.The learning rate was too low for the EOS row to changeD.The real EOS was masked too, so stopping was never learned

Discussion 0

Join free to post in the discussion.

Model never stops after SFT

Discussion 0

Related questions