AI Engineer (LLM & Generative AI)
LLM fundamentals questions
179 questions on LLM fundamentals for ai engineer (llm & generative ai)s, easiest first.
Easy 56
- What is a tokeneasy
- Effect of temperatureeasy
- What the context window limitseasy
- What a hallucination iseasy
- Temperature close to zeroeasy
- What top-k sampling doeseasy
- What LLM pretraining learnseasy
- Handling an HTTP 429 responseeasy
- Tokens in a 3,000-word reporteasy
- How a reply is generatedeasy
- A base model ignores the questioneasy
- What 8 billion parameters meanseasy
- Do examples in a prompt train the modeleasy
- Does the reply use context spaceeasy
- Hello and hello are not the sameeasy
- What happens to a token id firsteasy
- How the model decides to stopeasy
- Why models make things upeasy
- Open weights or open sourceeasy
- Running a model on a laptopeasy
- Keeping a system prompt secreteasy
- Emoji in a prompteasy
- Pretraining data compared with tuning dataeasy
- What greedy decoding iseasy
- What a frequency penalty doeseasy
- Presence penalty in briefeasy
- What a seed parameter promiseseasy
- Raising max tokens to get longer replieseasy
- How streamed replies arriveeasy
- Calling the LLM from the browsereasy
- Tracking tokens per requesteasy
- Which errors to retryeasy
- Checking prompt size in advanceeasy
- Model alias or dated versioneasy
- Every request fails after a deployeasy
- Reply stopped by a safety filtereasy
- Asking for five versions at onceeasy
- Long answers fail at 10 secondseasy
- Input or output, which costs moreeasy
- What TTFT measureseasy
- What p95 latency meanseasy
- Converting an API bill to rupeeseasy
- Cache entries do not last forevereasy
- Keeping data in the countryeasy
- Hosting your own modeleasy
- Choosing between two modelseasy
- A model retirement emaileasy
- Cost of a long system prompteasy
- Reasoning effort for easy taskseasy
- Will the provider train on our dataeasy
- The same question a thousand timeseasy
- Five summaries one after anothereasy
- Guarding against a surprise billeasy
- Reading a photo of a billeasy
- When you may not need retrievaleasy
- What fills the context windoweasy
Medium 71
- Making prompt caching workmedium
- Temperature changes probabilitiesmedium
- How many tokens top-p keepsmedium
- From logprobs to probabilitymedium
- Estimating monthly token costmedium
- Savings from a cached prefixmedium
- Why add jitter to backoffmedium
- Which rate limit binds firstmedium
- Token counts across two modelsmedium
- Why long sums go wrongmedium
- What self-attention doesmedium
- Doubling the input lengthmedium
- What the KV cache storesmedium
- How RLHF worksmedium
- What makes DPO differentmedium
- JSON versus plain text tokensmedium
- A larger tokenizer vocabularymedium
- Why transformers need positionsmedium
- Fine-tuned model lost a skillmedium
- Training a small model from a big onemedium
- How reasoning models are trainedmedium
- The model agrees too easilymedium
- Cutoff and release datemedium
- System prompt against user requestmedium
- An image in the promptmedium
- Feedback from AI with principlesmedium
- Pushing past the trained lengthmedium
- Why a 70B model is slowermedium
- The same sentence over and overmedium
- Why chat models skip beam searchmedium
- How many tokens min-p keepsmedium
- Probability after top-k filteringmedium
- Banning one word from the outputmedium
- Changing temperature and top-p togethermedium
- Sampling several answers and votingmedium
- Forcing output to fit a grammarmedium
- Parsing JSON while streamingmedium
- User clicks Stop mid-replymedium
- Provider overloaded at peak timemedium
- A bulk job hitting rate limitsmedium
- Limits that refill over timemedium
- Sending unsure labels to a personmedium
- The model says it is 95% suremedium
- Starting the reply for the modelmedium
- Room to read but not to writemedium
- Translating a 300-page manualmedium
- Total wait across five retriesmedium
- An empty reply from a reasoning modelmedium
- An API bill with 18% GSTmedium
- Overhead before the first tokenmedium
- When caching starts to paymedium
- A short prompt never cachesmedium
- Users in Mumbai, endpoint in the USmedium
- Steady high traffic and rate limitsmedium
- Which task needs a reasoning modelmedium
- Tokens for one imagemedium
- How to grade model outputsmedium
- Whole manual or retrieved partsmedium
- Picking matches from a listmedium
- Renting a GPU or paying per tokenmedium
- A few replies take a minutemedium
- Picking a model on cost and qualitymedium
- Latency numbers that look too goodmedium
- Summarising a 900-page reportmedium
- A batch job due in four hoursmedium
- Two calls where one would domedium
- Retrieved pages full of menusmedium
- Which feature costs the mostmedium
- Do cached prompts change the replymedium
- Trimming retrieved chunksmedium
- Doubling context lengthmedium
Hard 52
- Temperature 0 is not deterministichard
- Budgeting the context windowhard
- Fewer key-value headshard
- Training data for a 70B modelhard
- Estimating training computehard
- Upper limit on tokens per secondhard
- Calculating perplexityhard
- Odd replies from a self-hosted modelhard
- A suspiciously high benchmark scorehard
- Choosing a quantization levelhard
- Different answers at the same settingshard
- A 1M-token window in practicehard
- Probability of a two-token labelhard
- Weak replies in a regional languagehard
- Tuned model became too wordyhard
- Why training uses bfloat16hard
- Training on its own outputshard
- Entropy of a next-token choicehard
- Temperature and top-p togetherhard
- Schema mode still returned bad JSONhard
- Sampling ten code solutionshard
- Finishing a reply that was cut offhard
- The bot forgets its ruleshard
- Summarise or cut old turnshard
- Showing text before checks finishhard
- Every request waits through retrieshard
- A fallback model that misbehaveshard
- How many past turns fithard
- Penalties that break JSONhard
- Night job slows the live apphard
- Billed for a request that timed outhard
- Answers cut after one paragraphhard
- First token on a 100k prompthard
- Requests every eight minuteshard
- GPUs needed for peak traffichard
- Latency for a voice assistanthard
- Is fine-tuning a small model worth ithard
- Failover that breaks residency ruleshard
- Cheaper per token, but a bigger billhard
- Input cost when most tokens are cachedhard
- Testing a new model on live traffichard
- Caching and time to first tokenhard
- Sending a backup requesthard
- A new model that refuses too oftenhard
- Twenty reviews in one requesthard
- Fast in testing, slow at launchhard
- The full cost of running your ownhard
- Whole corpus in context versus RAGhard
- Extending RoPE by position interpolationhard
- Why YaRN scales frequencies unevenlyhard
- Reach of sliding-window attentionhard
- Passing the needle test is not enoughhard