easy5 pts · first answer countstransformerstransformer-architecture

Why transformers use residual connections

Each attention and feed-forward sublayer in a transformer adds its output back to its input (x + f(x)). What is the main benefit?

A.It doubles the vocabulary size without adding parametersB.It removes the need for any positional encoding in the modelC.It makes the model output the same token it received as inputD.Gradients flow easily through deep stacks, so many layers can be trained

Discussion 0

Join free to post in the discussion.

Why transformers use residual connections

Discussion 0

Related questions