hard20 pts · first answer countsinferencelatency

How speculative decoding helps

Which statement about speculative decoding is correct?

A.It skips layers of the large model, trading quality for speedB.It only speeds things up when sampling at high temperaturesC.A small draft model proposes tokens and the large model verifies them in one passD.It caches previous answers and returns them for similar prompts

Discussion 0

Join free to post in the discussion.

How speculative decoding helps

Discussion 0

Related questions