hard20 pts · first answer countsattentiongpu-performance

What FlashAttention changes

What does FlashAttention change compared with a naive attention implementation?

A.Attention approximated with low-rank matricesB.The softmax step is removed entirelyC.FLOPs drop from quadratic to linear in lengthD.Exact attention in tiles, never writing the full score matrix to HBM

Discussion 0

Join free to post in the discussion.

What FlashAttention changes

Discussion 0

Related questions