hard20 pts · first answer countsgpu-computinghardwaregpu-memory

H200 or H100 for long-context serving

An H200 has 141 GB HBM at about 4.8 TB/s; an H100 has 80 GB at about 3.35 TB/s. For a 70B FP8 model serving long contexts, why is the H200 attractive?

A.It has twice the FP8 tensor-core FLOPsB.It needs no NVLink for multi-GPU workC.It runs CUDA code without a driverD.More room for KV cache and faster decode reads

Discussion 0

Join free to post in the discussion.

H200 or H100 for long-context serving

Discussion 0

Related questions