hard20 pts · first answer countsgpu-computingparallelisminterconnect

Tensor parallel size beyond 8

On HGX nodes with 8 GPUs per NVLink domain, a team sets tensor parallel size to 16 across two nodes. Throughput falls sharply. Why?

A.TP size must always be a power of threeB.Per-layer TP traffic now crosses the networkC.GPUs cannot run more than 8 processes eachD.TP above 8 disables mixed precision

Discussion 0

Join free to post in the discussion.

Tensor parallel size beyond 8

Discussion 0

Related questions