medium10 pts · first answer countsgpu-computinggpu-memory

OOM with plenty of reserved memory

An OOM message says 6 GB is reserved but unallocated, yet a 2 GB allocation failed. Sequence lengths vary a lot. What setting often helps?

A.CUDA_DEVICE_ORDER=PCI_BUS_ID for all ranksB.TORCH_NCCL_ASYNC_ERROR_HANDLING=1 on startC.CUBLAS_WORKSPACE_CONFIG=:4096:8 for matmulsD.expandable_segments:True in the allocator config

Discussion 0

Join free to post in the discussion.

OOM with plenty of reserved memory

Discussion 0

Related questions