easy5 pts · first answer countstransformersattention

Query, key and value in attention

In self-attention, each token produces a query, a key and a value vector. Which description of their roles is right?

A.Query stores the token itself, key stores its position, value stores its frequencyB.Query and key are both copies of the input; value is the final output tokenC.Query asks what to look for, key is matched against it, value is what gets mixed inD.Query is used in training only, while key and value are used at inference only

Discussion 0

Join free to post in the discussion.

Query, key and value in attention

Discussion 0

Related questions