AI Engineer (LLM & Generative AI)
Decoding questions
29 questions on Decoding for ai engineer (llm & generative ai)s, easiest first.
Easy 12
- What the end-of-sequence token doeseasy
- What the final layer outputseasy
- What greedy decoding doeseasy
- Sampling from the top 40 tokenseasy
- What top-p sampling doeseasy
- Reply that ends mid-sentenceeasy
- Using a stop sequenceeasy
- Why chat replies arrive word by wordeasy
- What a repetition penalty is foreasy
- Making sampled outputs repeatableeasy
- What logprobs tell youeasy
- What beam search keepseasy
Medium 10
- Counting tokens in the nucleusmedium
- Probabilities after top-kmedium
- Which tokens min-p keepsmedium
- Beam search for open-ended chatmedium
- Presence versus frequency penaltymedium
- How speculative decoding speeds things upmedium
- Guaranteeing valid JSON outputmedium
- Banning a token with logit biasmedium
- Different answers at temperature 0medium
- Sampling several reasoning pathsmedium