medium10 pts · first answer countsgpu-computingcheckpointing

Training pauses while saving

Each checkpoint of a large model pauses training for 6 minutes while it is written to storage. What reduces the pause the most?

A.Saving checkpoints twice as often as nowB.Using a smaller learning rateC.Async checkpointing in the backgroundD.Writing checkpoints as JSON text

Discussion 0

Join free to post in the discussion.