medium10 pts · first answer countsdeep-learninglearning-rate-schedulestraining
Reading an LR range test
In a learning-rate range test, you raise the LR each batch from 1e-7 to 1 and plot loss. Loss falls fastest around 1e-3 and explodes near 1e-1. What is a sensible choice?
Discussion 0
Join free to post in the discussion.
Related questions
- When early stopping triggersMachine Learning Engineer
- Optimizer steps in one epochMachine Learning Engineer
- Parameters in nn.Linear(10, 5)Machine Learning Engineer
- Why sigmoid slows deep trainingMachine Learning Engineer
- Range of tanhMachine Learning Engineer
- Cross-entropy loss by handMachine Learning Engineer
- Telling autograd to track a tensorMachine Learning Engineer
- Why wrap inference in no_gradMachine Learning Engineer