Machine Learning Engineer
Fine-tuning questions
165 questions on Fine-tuning for machine learning engineers, easiest first.
Easy 48
- What full fine-tuning changeseasy
- Why begin from a pretrained modeleasy
- Training only a new headeasy
- Learning rate for fine-tuningeasy
- Shape of SFT training dataeasy
- Reading train and validation losseasy
- Meaning of PEFTeasy
- What the LoRA rank r setseasy
- Merging a LoRA adaptereasy
- What QLoRA adds to LoRAeasy
- Choosing the tokenizereasy
- Prompt format after SFTeasy
- Purpose of instruction tuningeasy
- First step before fine-tuningeasy
- Role of lora_dropouteasy
- Order of classic RLHF stageseasy
- What a reward model returnseasy
- Format of preference dataeasy
- Basic idea of distillationeasy
- Why soft targets help a studenteasy
- Temperature in distillationeasy
- Training on a bigger model's answerseasy
- Using another provider's outputseasy
- Labellers disagree on answerseasy
- Meaning of RLAIFeasy
- Teaching safe refusalseasy
- What reward hacking meanseasy
- Purpose of the KL penaltyeasy
- Meaning of the alignment taxeasy
- Picking a student modeleasy
- Library for SFT and DPO runseasy
- Adding LoRA with the PEFT libraryeasy
- Splitting a skewed label seteasy
- Why save checkpointseasy
- Fine-tuning BERT for classificationeasy
- Data for tuning an embedding modeleasy
- Speech model misses local accentseasy
- Teaching a domain vocabularyeasy
- Customer data in training setseasy
- File format for hosted fine-tuningeasy
- How many epochs for LLM SFTeasy
- Comparing many training runseasy
- Making runs repeatableeasy
- New classes for an image modeleasy
- Small data for a narrow taskeasy
- What to record for each adaptereasy
- Early stopping patienceeasy
- What BitFit trainseasy
Medium 65
- Trainable parameters in a LoRA adaptermedium
- What lora_alpha doesmedium
- How LoRA is initialisedmedium
- Choosing LoRA target modulesmedium
- Training only on the responsemedium
- Label value -100 in Hugging Facemedium
- Packing short examples togethermedium
- Batch size with accumulationmedium
- Why use learning-rate warmupmedium
- Why QLoRA uses NF4medium
- Double quantisation in QLoRAmedium
- What paged optimizers solvemedium
- Adding special tokensmedium
- Model never stops after SFTmedium
- Quality versus quantity of SFT datamedium
- Duplicates in training datamedium
- Typical LoRA learning ratemedium
- What prompt tuning trainsmedium
- Adapters versus LoRA at inferencemedium
- Cosine learning-rate schedulemedium
- Evaluating a new fine-tunemedium
- Loss for training a reward modelmedium
- Win probability from a reward gapmedium
- Effect of beta in DPOmedium
- Choosing the DPO reference modelmedium
- What makes ORPO differentmedium
- Training on thumbs up and downmedium
- Why GRPO is cheaper than PPOmedium
- Rewards from automatic checksmedium
- Clipping in the PPO objectivemedium
- Answers grow longer after RLmedium
- Distillation loss on logitsmedium
- Combining label and teacher lossesmedium
- Distilling without teacher logitsmedium
- Why on-policy distillation helpsmedium
- Best-of-n then fine-tunemedium
- Optimising a proxy reward too hardmedium
- Building preference pairsmedium
- Chosen and rejected almost equalmedium
- Starting point for a reward modelmedium
- Fair pairwise LLM-as-judgemedium
- Different rates for different layersmedium
- Gradual unfreezingmedium
- Classifier ignores the rare classmedium
- Hard negatives for embeddingsmedium
- Batch size in contrastive tuningmedium
- DreamBooth forgets the general classmedium
- Choosing a metric for free textmedium
- Releasing a new adapter versionmedium
- Hindi and Arabic use more tokensmedium
- Synthetic examples all look alikemedium
- Reliable JSON from a small modelmedium
- Training a model to call toolsmedium
- Cost of a hosted fine-tunemedium
- Time for a self-hosted runmedium
- Long examples cut at max lengthmedium
- OOM after doubling sequence lengthmedium
- Making LLM-judge scores reliablemedium
- Training data sorted by categorymedium
- Order of domain training stepsmedium
- Is the fine-tune worth itmedium
- How IA3 adapts a modelmedium
- Removing a 3,000-token system promptmedium
- Adding tokens for an Indic scriptmedium
- LoRA cannot learn new tokensmedium
Hard 52
- Fine-tuning a CNN with batch size 2hard
- Why scale distillation loss by T squaredhard
- How DPO differs from PPO-based RLHFhard
- Fine-tuned model forgets general skillshard
- Base weights for 13B in 4-bithard
- Adam state for a LoRA adapterhard
- Merging a QLoRA adapterhard
- Rank 256 on 800 exampleshard
- What DoRA changeshard
- Why rank-stabilised LoRA existshard
- Counting optimizer stepshard
- Loss on multi-turn chatshard
- NaN loss in fp16 fine-tuninghard
- Loss jumps then output is gibberishhard
- One base, several taskshard
- Checkpointing with frozen inputshard
- Training loss of 0.02hard
- Weight decay on LoRAhard
- Averaging loss with accumulationhard
- DPO loss for one pairhard
- Chosen likelihood falls during DPOhard
- Policy writes nonsense that scores wellhard
- Teacher and student tokenizers differhard
- Softmax at T = 2hard
- Off-policy preference datahard
- Advantages in a GRPO grouphard
- Distilling step-by-step reasoninghard
- Safety after a small fine-tunehard
- Process and outcome reward modelshard
- Less varied answers after RLHFhard
- Running DPO in roundshard
- DPO directly on a base modelhard
- Pairs where both answers are equalhard
- Reward model held-out accuracyhard
- Merging two fine-tunes of one basehard
- Why TIES merging trims and elects signshard
- Averaging several fine-tuned runshard
- Memory for full-vocabulary logitshard
- Fine-tuning with differential privacyhard
- Better perplexity, worse task scorehard
- Works only with one exact wordinghard
- Rank sweep resultshard
- Balancing a multilingual data mixhard
- Finding mislabelled rowshard
- Accuracy loss at 4-bit for servinghard
- Model repeats a customer recordhard
- Exponential moving average of weightshard
- Extending context during fine-tuninghard
- Splitting chat logs for evaluationhard
- Fine-tuned small model versus APIhard
- Offline win, online questionhard
- LoRA on a mixture-of-experts modelhard