medium10 pts · first answer countsgpu-computingcollectives
Traffic in ring all-reduce
In ring all-reduce of a tensor of size S across N GPUs, how much data does each GPU send in total?
Discussion 0
Join free to post in the discussion.
In ring all-reduce of a tensor of size S across N GPUs, how much data does each GPU send in total?
Join free to post in the discussion.