easy5 pts · first answer countsquantisationinference-optimisation

Why quantise a model

What is the main reason to quantise a model to int8 or 4-bit for serving?

A.To improve accuracy on new dataB.To make training converge fasterC.To cut memory use and often speed up inference, at a small accuracy costD.To make the model easier to explain

Discussion 0

Join free to post in the discussion.

Why quantise a model

Discussion 0

Related questions