hard20 pts · first answer countsdeep-learningoptimisersmaths
Momentum update by hand
SGD with momentum uses v ← μ·v + g, then w ← w − lr·v. With μ = 0.9, current v = 1.0, gradient g = 0.5, lr = 0.1 and w = 3.0, what is the new w?
Discussion 0
Join free to post in the discussion.
Related questions
- Parameters in nn.Linear(10, 5)Machine Learning Engineer
- Why sigmoid slows deep trainingMachine Learning Engineer
- Range of tanhMachine Learning Engineer
- Cross-entropy loss by handMachine Learning Engineer
- Telling autograd to track a tensorMachine Learning Engineer
- Why wrap inference in no_gradMachine Learning Engineer
- Logging loss without leaking memoryMachine Learning Engineer
- Feeding a single image to a CNNMachine Learning Engineer