medium10 pts · first answer countsmixed-precisiontraining
Why bf16 is easier than fp16
Why is bf16 mixed precision usually easier to use than fp16 for training?
Discussion 0
Join free to post in the discussion.
Related questions
- When early stopping triggersMachine Learning Engineer
- Optimizer steps in one epochMachine Learning Engineer
- Training loop without zero_gradMachine Learning Engineer
- Why use learning rate warmupMachine Learning Engineer
- Loss turns NaN in an RNNMachine Learning Engineer
- Why fp16 needs loss scaling but bf16 usually does notMachine Learning Engineer