medium10 pts · first answer countstransformersmixture-of-expertsinference

Active parameters in an MoE model

An MoE model has 2B shared parameters (attention, embeddings) and 8 experts of 5B parameters each. Each token uses the top 2 experts. How many parameters are active per token, and how many must be in memory?

A.42B active, 12B in memoryB.12B active, 12B in memoryC.12B active, 42B in memoryD.7B active, 42B in memory

Discussion 0

Join free to post in the discussion.

Active parameters in an MoE model

Discussion 0

Related questions