medium10 pts · first answer countstransformerstraining

What gradient checkpointing trades

Fine-tuning runs out of GPU memory because of activations. You turn on gradient checkpointing. What is the trade-off?

A.Less memory and faster training, with lower final accuracyB.More memory used, but each step becomes fasterC.Smaller checkpoints on disk, but no change in GPU memoryD.Less activation memory, but about 20–35% more compute from recomputing

Discussion 0

Join free to post in the discussion.

What gradient checkpointing trades

Discussion 0

Related questions