medium10 pts · first answer countsllm-fundamentalsllm-basics

Why transformers need positions

Without any positional information, self-attention would treat "dog bites man" and "man bites dog" too similarly. How do LLMs fix this?

A.They sort all tokens in alphabetical order firstB.They only allow sentences with fewer than five wordsC.They add position signals, for example rotary embeddings (RoPE)D.They read each sentence twice, forward and then backward

Discussion 0

Join free to post in the discussion.

Why transformers need positions

Discussion 0

Related questions