hard20 pts · first answer countsquantisation

Accuracy fell after int8 quantisation

After post-training int8 quantisation a model loses 6 points of accuracy. Analysis shows a few activation channels have huge outlier values. What is the best response?

A.Quantise to 2-bit insteadB.Use finer-grained scales (per-channel or per-group) or keep sensitive layers in higher precision, and recalibrate on representative dataC.Remove the calibration dataD.Increase the batch size

Discussion 0

Join free to post in the discussion.

Accuracy fell after int8 quantisation

Discussion 0

Related questions