Skip to content
RED SAGSkills
easy5 pts · first answer countstransformersinference

Memory for 7B weights in fp16

Roughly how much GPU memory do the weights of a 7-billion-parameter model need in fp16, before any KV cache or activations?

Choose one answer

Discussion 0

Join free to post in the discussion.

Related questions

More Transformers and LLM internals questionsMore Inference questions