hard20 pts · first answer countsmodel-servingspeculative-decoding

Speculation without a separate draft model

Methods like Medusa and EAGLE speed up decoding without a separate small draft model. How?

A.They skip half the layers for every tokenB.They lower the precision of every second tokenC.They cache whole answers from earlier usersD.Small heads on the target model guess future tokens

Discussion 0

Join free to post in the discussion.

Speculation without a separate draft model

Discussion 0

Related questions