hard20 pts · first answer countsgpu-computinggpu-memory
What grows activation memory
For a transformer in training, stored activation memory grows roughly in proportion to which product (ignoring attention scores)?
Discussion 0
Join free to post in the discussion.