41. Parameters in a multi-head attention block
A multi-head attention block has d_model = 512 and 8 heads, with Q, K, V and output projections and no biases. How many parameters does it have?
- A.262,144
- B.1,048,576
- C.2,097,152
- D.8,388,608
Show answer
Answer: B. 1,048,576
Each of the four projections is a 512 x 512 matrix, so 4 * 262,144 = 1,048,576. Splitting into 8 heads of size 64 only reshapes these projections; it does not add parameters. Distractors come from counting one projection or multiplying by the head count.
42. Fine-tuned model forgets general skills
After full fine-tuning on a narrow support dataset, your LLM handles support tickets well but has lost general reasoning and formatting skills. What is the best response?
- A.Train for more epochs on the same narrow data
- B.Raise the learning rate tenfold and retrain
- C.Re-initialise the pre-trained weights and start over
- D.Mix in general data, lower the LR or use adapters, and track general evals
Show answer
Answer: D. Mix in general data, lower the LR or use adapters, and track general evals
This is catastrophic forgetting: large updates on narrow data overwrite general abilities. Replaying some general data, using smaller learning rates or fewer epochs, and parameter-efficient methods like LoRA limit how far weights move. A regression suite of general tasks catches it before release.
43. Why long contexts eat GPU memory
When serving a self-hosted LLM, GPU memory runs out as users send longer prompts and you increase concurrency. What mainly grows with sequence length and batch size?
- A.The model weights
- B.The KV cache: stored key and value tensors for every token, layer and request
- C.The tokenizer vocabulary
- D.The embedding table
Show answer
Answer: B. The KV cache: stored key and value tensors for every token, layer and request
Model weights are fixed in size, but the KV cache stores attention keys and values for each token in each layer for every active request, so it grows roughly with sequence length times batch size. Techniques like paged attention, KV cache quantisation and grouped-query attention reduce this cost.
44. Extending RoPE by position interpolation
A model was trained with RoPE at 4K context. To use it at 16K, a team applies position interpolation before a short fine-tune. What does position interpolation do?
- A.Scales positions down by 4 so 16K positions map into the trained 0–4K range
- B.Repeats the 4K position pattern four times across the 16K sequence
- C.Replaces RoPE with learned absolute embeddings for positions over 4K
- D.Drops every fourth token so the sequence fits in 4K positions
Show answer
Answer: A. Scales positions down by 4 so 16K positions map into the trained 0–4K range
Rotation angles past 4K were never seen in training, so extrapolating to them breaks attention. Interpolation multiplies each position by 4K/16K, keeping all angles inside the trained range, and a short fine-tune adapts the model to the finer spacing. Methods like NTK-aware scaling and YaRN refine this idea.
45. Decoder with no positional encoding
Research shows a decoder-only transformer trained with no positional encoding at all (NoPE) can still learn word order. How is that possible?
- A.The tokenizer secretly adds a position number to each token id
- B.The feed-forward layers sort the tokens back into order
- C.Under the causal mask each position sees a different count of tokens
- D.It cannot; such models produce identical outputs for shuffled text
Show answer
Answer: C. Under the causal mask each position sees a different count of tokens
With a causal mask, position 1 attends to one token, position 100 to a hundred, so attention averages differ systematically with position. The model can learn to read this signal and recover order. An encoder with full bidirectional attention has no such asymmetry and really is order-blind without positions.
46. Sliding cache that drops the first tokens
To stream very long chats, you keep only the latest 4,000 tokens in the KV cache. As soon as the earliest tokens are evicted, output quality collapses. What is a known fix?
- A.Always keep the first few tokens, which act as attention sinks
- B.Lower the temperature so the model depends less on old tokens
- C.Evict the most recent tokens instead of the oldest ones
- D.Double the number of attention heads at inference time
Show answer
Answer: A. Always keep the first few tokens, which act as attention sinks
Many heads park excess attention weight on the first tokens, which act as "attention sinks" because softmax weights must sum to 1. Evicting them shifts that weight onto random tokens and breaks the model. StreamingLLM showed that keeping a handful of initial tokens plus a recent window restores stable quality.
47. Subtracting the max in softmax
Production softmax kernels compute exp(x − max(x)) instead of exp(x). Why is this safe and why is it done?
- A.Softmax is unchanged by subtracting a constant, and it stops exp() from overflowing
- B.It makes the largest probability exactly 1 so decoding is greedy
- C.It changes the distribution slightly but makes training faster
- D.It is only needed when the vocabulary has more than 65,536 tokens
Show answer
Answer: A. Softmax is unchanged by subtracting a constant, and it stops exp() from overflowing
Softmax(x) = softmax(x − c) for any constant c, because exp(−c) cancels in the numerator and denominator. Using c = max(x) makes every exponent ≤ 0, so no value overflows to infinity, which matters a lot in fp16 and bf16. The output is mathematically the same.
48. NaNs from a fully masked row
Training is fine until a batch contains an example that is entirely padding. The loss becomes NaN. Your mask fills masked scores with −inf. What is the likely cause?
- A.−inf is converted to +inf in bf16, so pad tokens get all the weight
- B.Padding tokens have no embeddings, so the lookup returns NaN
- C.The learning rate is too high only for padded examples
- D.A row where every score is −inf gives softmax 0/0, which is NaN
Show answer
Answer: D. A row where every score is −inf gives softmax 0/0, which is NaN
If every key in a row is masked, softmax computes exp(−inf)/Σexp(−inf) = 0/0 = NaN, and the NaN spreads through the network. Fixes include using a large finite negative number like the dtype minimum, skipping fully masked rows, or making sure every query can attend to at least one token.
49. Prompt that ends with a space
A few-shot prompt ends with "Answer: " (with a trailing space). Completions are worse than when the prompt ends with "Answer:". Why?
- A.Trailing spaces are stripped by the API, so the prompt is cut short
- B.The space is read as an end-of-sequence marker and stops generation
- C.Spaces count double against the context window, so examples are dropped
- D.Words are tokens with a leading space, so the model must now emit a rare space-less token
Show answer
Answer: D. Words are tokens with a leading space, so the model must now emit a rare space-less token
In GPT-style tokenizers the space usually belongs to the next word, as in " Paris". If the prompt already ends with a space token, the model must continue with "Paris" without a leading space, which is a much rarer token in that spot. This boundary effect is why some libraries do "token healing".
50. Merge or keep LoRA adapters separate
You have one base model and 40 customer-specific LoRA adapters. Traffic mixes all customers in each batch. What serving approach fits best?
- A.Merge all 40 adapters into the base weights by adding them together
- B.Keep adapters unmerged and use a server that applies the right one per request
- C.Deploy 40 full merged copies of the model on separate GPUs
- D.Retrain one adapter on all customers' data and drop the rest
Show answer
Answer: B. Keep adapters unmerged and use a server that applies the right one per request
Merging (W + BA) removes adapter latency but only works for one adapter per copy of the model. Multi-LoRA servers such as vLLM or LoRAX keep one base in memory and apply each request's small adapter in batched kernels. Summing 40 adapters would blend customers' behaviour together.
51. Teaching new facts by fine-tuning
A bank wants its chatbot to know this month's changing interest rates. A teammate proposes monthly LoRA fine-tunes on the rate sheet. What is the main concern?
- A.LoRA cannot be used on any model larger than 1B parameters
- B.Fine-tuning injects facts poorly; retrieving the sheet fits better
- C.Fine-tuning always erases the model's ability to speak English
- D.Rates are numbers, and transformers cannot store numbers at all
Show answer
Answer: B. Fine-tuning injects facts poorly; retrieving the sheet fits better
Fine-tuning is good at teaching format, style and behaviour, but models absorb new facts slowly and may blend old and new values. Studies also link training on unfamiliar facts to more hallucination. For facts that change often, retrieving the current rate sheet at query time is more accurate and auditable.
52. Token probabilities after RLHF
You use a chat model's token probabilities as confidence scores for multiple-choice answers. They were well calibrated on the base model but are overconfident on the RLHF-tuned model. Why?
- A.RLHF changes the tokenizer, so the probabilities refer to new tokens
- B.Preference tuning pushes outputs towards confident answers, hurting calibration
- C.Base models are never calibrated, so the earlier result was luck
- D.The KL penalty forces all probabilities to equal 0.5
Show answer
Answer: B. Preference tuning pushes outputs towards confident answers, hurting calibration
Pretraining with cross-entropy tends to produce calibrated next-token probabilities. RLHF optimises for rewarded answers rather than likelihood, and the distribution often becomes sharper, so 90% stated confidence may be right far less often. Recalibrate on held-out data, for example with temperature scaling.
53. Fine-tuned model never stops talking
After SFT, your model keeps generating past the answer until max_tokens. The training code sets pad_token = eos_token and masks all pad positions out of the loss. What is the bug?
- A.The real EOS was masked too, so the model never learned to emit it
- B.max_tokens is ignored whenever pad and EOS share an id
- C.The learning rate was too low to learn short answers
- D.Masking pad tokens makes the model forget the chat template
Show answer
Answer: A. The real EOS was masked too, so the model never learned to emit it
If PAD and EOS share an id and every pad-id position is masked, the true end-of-reply EOS gets masked as well. The model never sees a loss for stopping, so it learns to keep going. Use a separate pad token or mask by attention mask so the first EOS keeps its label.
54. Sizing GPUs for an MoE model
A teammate says: "This MoE model has 47B total but only 13B active parameters, so it fits on the same GPU as a 13B dense model." What is wrong with this?
- A.Nothing; only active parameters ever need to be loaded
- B.MoE models need more memory than their total parameter count
- C.All 47B must be resident, since any token may route to any expert
- D.Active parameters are always larger than the total in practice
Show answer
Answer: C. All 47B must be resident, since any token may route to any expert
Active parameters set the compute per token, but routing changes from token to token, so every expert's weights must be available. In fp16 the 47B model needs about 94 GB just for weights versus about 26 GB for 13B dense. Offloading experts is possible but slows decoding a lot.
55. Greedy decoding stuck in a loop
A small model with greedy decoding writes "The product is great. The product is great. The product is great..." Why does greedy decoding fall into such loops?
- A.Greedy decoding has a bug that copies the previous sentence
- B.The model ran out of vocabulary and reused old tokens
- C.The stop sequence was too long for the model to produce
- D.Each repeat makes repeating again the most likely next step
Show answer
Answer: D. Each repeat makes repeating again the most likely next step
Attention can copy patterns from context, so each repetition makes the next repetition more probable, a self-reinforcing loop. Greedy decoding never takes a lower-probability exit. Sampling with top-p, repetition or frequency penalties, or n-gram blocking breaks the cycle.
56. Long prompts slowing other users
On a shared server, whenever someone sends a 100K-token prompt, every other user's streaming output pauses for a few seconds. Which technique targets this?
- A.Raising the temperature for the long-prompt request only
- B.Turning off the KV cache for every request on the server
- C.Chunked prefill, mixing prompt chunks with decode steps
- D.Using beam search for the shorter requests
Show answer
Answer: C. Chunked prefill, mixing prompt chunks with decode steps
A huge prefill can fill the GPU for seconds, and decode steps for other requests wait behind it. Chunked prefill processes the long prompt in pieces mixed with ongoing decode work, smoothing inter-token latency. Some systems go further and run prefill and decode on separate GPU pools.
57. JSON parse errors on long outputs
About 3% of your JSON responses fail to parse, all of them for the longest documents. The model otherwise follows the schema well. What should you check first?
- A.Whether the finish reason is "length", meaning the output was cut off
- B.Whether the temperature is above 0.2 for those requests
- C.Whether the model was updated by the provider that week
- D.Whether the JSON keys use camelCase or snake_case
Show answer
Answer: A. Whether the finish reason is "length", meaning the output was cut off
Long inputs often need long outputs, and hitting max_tokens truncates the JSON mid-object even under schema constraints. Always check the finish reason before parsing. Raise the limit, shorten fields, or split the task when outputs are long.
58. Ordering prompts for cache hits
Your prompt is: current date and user name, then a 5,000-token policy manual, then the question. Prompt caching is on, but cache hits are almost zero. What fix gives the most benefit?
- A.Shorten the manual to under 1,000 tokens so it fits the cache
- B.Move the fixed manual to the very start and put the date and name after it
- C.Turn off streaming so the cache can store the full output
- D.Send the manual as a separate request before every question
Show answer
Answer: B. Move the fixed manual to the very start and put the date and name after it
Prefix caching only reuses an exact token prefix. With the changing date and name at the top, every prompt differs from the first token, so nothing matches. Putting stable content first and variable content last lets every request reuse the manual's cached KV.
59. GPU at low utilisation during decoding
During single-user decoding, GPU compute utilisation is about 5%, yet tokens/s will not go higher. Adding more concurrent users barely slows each one at first. What explains this?
- A.The GPU is throttling its clock because it is too hot
- B.The tokenizer runs on the GPU and blocks the model
- C.Decode waits on weight reads, which extra sequences share
- D.Each user is served by a separate idle GPU core
Show answer
Answer: C. Decode waits on weight reads, which extra sequences share
At batch size 1, each step reads all weights for one matrix-vector product, so arithmetic intensity is tiny and the GPU waits on memory. Adding sequences turns it into a matrix-matrix product that reuses the same bytes, so throughput rises almost for free until compute or KV-cache memory becomes the limit.
60. State-space models versus attention
Mamba-style state-space models keep a fixed-size state instead of a growing KV cache. What is the main trade-off compared with attention?
- A.Better exact recall, but memory that grows faster than attention's
- B.Constant memory per token, but weaker exact recall of details far back
- C.No training needed, but they only work on short inputs
- D.Faster prefill, but they cannot generate text at all
Show answer
Answer: B. Constant memory per token, but weaker exact recall of details far back
A fixed state means generation costs the same per token regardless of length, which suits very long inputs. But everything must be compressed into that state, so copying an exact phone number from 50K tokens ago is harder than for attention, which can look it up directly. Hybrid models mix both layer types.
