medium10 pts · first answer countstransformersattention
Computing an attention output
One query has scaled scores [2, 0] against two keys. Their value vectors are v1 = [1, 0] and v2 = [0, 1]. What is the attention output, to two decimals? (e² ≈ 7.39)
Discussion 0
Join free to post in the discussion.
Related questions
- What LLM pretraining learnsAI Engineer (LLM & Generative AI)
- How BPE builds its vocabularyAI Engineer (LLM & Generative AI)
- Tokens per English wordAI Engineer (LLM & Generative AI)
- Why Hindi text uses more tokensAI Engineer (LLM & Generative AI)
- What the end-of-sequence token doesAI Engineer (LLM & Generative AI)
- What the embedding layer doesAI Engineer (LLM & Generative AI)
- Query, key and value in attentionAI Engineer (LLM & Generative AI)
- Why transformers need position informationAI Engineer (LLM & Generative AI)