hard20 pts · first answer countsattentiontransformers

What the causal mask prevents

During training, a decoder-only transformer processes a whole sequence in parallel. What does the causal attention mask do, and why is it needed?

A.It hides padding tokens so batches can have different lengthsB.It blocks attention to later positions, so the model cannot see the token it must predictC.It drops random attention weights during training to reduce overfittingD.It limits attention to a fixed window of nearby tokens to save memory

Discussion 0

Join free to post in the discussion.

What the causal mask prevents

Discussion 0

Related questions