Skip to content
RED SAGSkills
hard20 pts · first answer countsgpu-computingcollectives

Rings versus trees at large scale

For small messages across thousands of GPUs, NCCL may choose a tree algorithm instead of a ring. Why?

Choose one answer

Discussion 0

Join free to post in the discussion.

Related questions

More GPUs and distributed training questions