easy5 pts · first answer countstransformersdecoding

Sampling from the top 40 tokens

With top-k sampling and k = 40, how is the next token chosen?

A.Generate 40 tokens and keep the one with the highest scoreB.Keep the 40 most likely tokens, renormalise, then sample from themC.Sample from the whole vocabulary 40 times and take a voteD.Pick the 40th most likely token every time

Discussion 0

Join free to post in the discussion.

Sampling from the top 40 tokens

Discussion 0

Related questions