easy5 pts · first answer countstransformersattention

Why use several attention heads

Why does a transformer layer split attention into several heads instead of using one big head?

A.Each head can learn a different pattern, such as syntax or nearby wordsB.More heads always mean many more parameters and so more knowledgeC.Each head handles a different language in multilingual modelsD.Heads are needed so that the softmax can be computed on a CPU

Discussion 0

Join free to post in the discussion.

Why use several attention heads

Discussion 0

Related questions