hard20 pts · first answer countsdistributed-trainingmemory-optimization
Activation checkpointing trade-off
What does activation (gradient) checkpointing trade?
Discussion 0
Join free to post in the discussion.
Related questions
- Effective batch size across GPUsMachine Learning Engineer
- Model does not fit in GPU memoryMachine Learning Engineer
- What ZeRO stage 3 shardsMachine Learning Engineer
- Training on spot GPUsMLOps Engineer
- Speeding up training on 8 GPUsMLOps Engineer
- Where to use tensor parallelismMLOps Engineer
- Memory to fully fine-tune 7B with AdamMLOps Engineer
- Loss spike after resuming trainingMLOps Engineer