hard20 pts · first answer countsgpu-memorydistributed-training
Memory to fully fine-tune 7B with Adam
You fully fine-tune a 7B model with Adam in mixed precision (16-bit weights and gradients, fp32 master weights and two fp32 Adam moments). Ignoring activations, roughly how much memory is needed?
Discussion 0
Join free to post in the discussion.
Related questions
- Memory for 7B parameters in fp16MLOps Engineer
- KV cache size for one long requestMLOps Engineer
- Training on spot GPUsMLOps Engineer
- Speeding up training on 8 GPUsMLOps Engineer
- Where to use tensor parallelismMLOps Engineer
- KV cache with grouped-query attentionMLOps Engineer
- Loss spike after resuming trainingMLOps Engineer
- Effective batch size across GPUsMachine Learning Engineer