hard20 pts · first answer countsquantization

INT8 activations hurt accuracy

An LLM keeps its quality with 8-bit weight-only quantisation, but quality drops sharply when activations are also quantised to INT8. What is the most likely reason?

A.A few outlier activation channels dominate the quantisation scaleB.INT8 cannot represent negative numbersC.Weights must be quantised before pre-trainingD.The tokenizer must be retrained for INT8

Discussion 0

Join free to post in the discussion.

INT8 activations hurt accuracy

Discussion 0

Related questions