hard20 pts · first answer countsllm-fundamentalssampling

Different answers at the same settings

With greedy decoding, the same prompt sometimes gives a different answer when the server is busy. Which explanation is most accurate?

A.The model learns from each request and changes its weights liveB.Batching changes float maths order and flips near-tied tokensC.Greedy decoding picks tokens at random when the server is busyD.The tokenizer uses a different vocabulary for each user

Discussion 0

Join free to post in the discussion.

Different answers at the same settings

Discussion 0

Related questions