easy5 pts · first answer countstransformersattention

Why a padding mask is needed

You batch sentences of different lengths and pad the short ones with a PAD token. Why must you also pass an attention mask?

A.So the PAD tokens are converted into spaces before tokenizationB.So the model learns to predict PAD tokens as the next wordC.So real tokens do not attend to the PAD positions and change their outputsD.So that the batch can be sorted from shortest to longest

Discussion 0

Join free to post in the discussion.

Why a padding mask is needed

Discussion 0

Related questions