medium10 pts · first answer countstransformersattentiontransformer-architecture

Where cross-attention gets its inputs

In an encoder-decoder transformer such as T5, the decoder has a cross-attention layer. Where do its queries and its keys/values come from?

A.Queries from the encoder output; keys and values from the decoderB.Queries from the decoder; keys and values from the encoder outputC.Queries, keys and values all from the decoder's previous layerD.Queries, keys and values all from the raw input embeddings

Discussion 0

Join free to post in the discussion.

Where cross-attention gets its inputs

Discussion 0

Related questions