medium10 pts · first answer countsgpu-computingmixed-precision

Training in FP8

Libraries such as NVIDIA Transformer Engine let H100 GPUs train with FP8 matmuls. What do they track to keep FP8 usable?

A.A separate learning rate for each FP8 layerB.A copy of the dataset in FP8C.The CPU temperature during trainingD.Per-tensor scale factors from recent max values

Discussion 0

Join free to post in the discussion.

Training in FP8

Discussion 0

Related questions