Fine-Tuning interview questions and answers

1. Learning rate for fine-tuning

Compared with the learning rate used to pretrain a model, the learning rate for full fine-tuning is usually:

1. A.Larger, so the model forgets its old task quickly
2. B.Exactly the same, because the optimizer is the same
3. C.Zero for all layers except the last one
4. D.Smaller, so useful pretrained weights are not wiped out

2. Shape of SFT training data

For supervised fine-tuning (SFT) of a chat model, what does each training example usually contain?

1. A.A prompt with two ranked responses
2. B.Only raw text with no structure
3. C.A prompt and the ideal response to it
4. D.A response with a numeric reward score

3. Meaning of PEFT

What is the main idea of parameter-efficient fine-tuning (PEFT) methods such as LoRA and prompt tuning?

1. A.Train every parameter but with fewer epochs
2. B.Remove half the layers before training
3. C.Train on only a small random sample of the dataset
4. D.Train a small number of parameters and keep the rest frozen

4. Purpose of instruction tuning

A raw pretrained base model tends to continue your text instead of answering it. What does instruction tuning mainly add?

1. A.Knowledge of facts the base model never saw
2. B.The habit of following instructions and answering
3. C.Support for longer context windows
4. D.A smaller vocabulary for faster decoding

5. Role of lora_dropout

In a PEFT LoraConfig, what does setting lora_dropout=0.05 do?

1. A.Randomly drops adapter inputs during training only
2. B.Removes 5% of base model layers permanently
3. C.Skips 5% of the training examples in each epoch
4. D.Lowers the learning rate by 5% after every step

6. Order of classic RLHF stages

The classic RLHF pipeline used for early chat models has three stages. In what order do they run?

1. A.SFT, then reward model, then RL against the reward
2. B.Reward model, then pretraining, then SFT
3. C.RL first, then SFT, then the reward model
4. D.Distillation, then SFT, then pretraining

7. What a reward model returns

In RLHF, what does the reward model output for a prompt and a response?

1. A.A rewritten, improved version of the response
2. B.A single number for how good the response is
3. C.The next token the policy should produce
4. D.A list of training examples to add

8. Temperature in distillation

In distillation, the teacher's logits are divided by a temperature T greater than 1 before softmax. What does that do?

1. A.Makes the probability distribution flatter
2. B.Makes the top class probability closer to 1
3. C.Changes which class has the highest score
4. D.Removes the smallest classes entirely

9. Adding LoRA with the PEFT library

What does get_peft_model(model, LoraConfig(...)) do in Hugging Face PEFT?

from peft import LoraConfig, get_peft_model
config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(model, config)
model.print_trainable_parameters()

1. A.Quantises the whole model to 4-bit
2. B.Adds LoRA layers and freezes the base weights
3. C.Deletes all layers not named in the list
4. D.Downloads a ready-trained adapter

10. Splitting a skewed label set

You fine-tune a classifier on complaints where "fraud" is only 3% of rows. How should you split train and test sets?

1. A.Put all of the fraud rows into the test set
2. B.Take the first 80% of rows as the training set
3. C.Stratify by label so each split keeps the 3% share
4. D.Use only fraud rows for training

11. Why save checkpoints

Why should a multi-hour fine-tuning run save checkpoints regularly?

1. A.Because the GPU deletes weights every hour
2. B.So the tokenizer can be retrained each time
3. C.To make every training step run faster
4. D.To resume after a crash and to pick the best model

12. Teaching a domain vocabulary

A legal-tech firm has 2 GB of unlabelled Indian court judgments and wants the model to understand that language better. Which method fits this data?

1. A.DPO on pairs of judgments
2. B.Continued pretraining on the raw text
3. C.Training a reward model on the files
4. D.Prompt tuning with one example

13. Customer data in training sets

Your SFT set is built from real support chats that include phone numbers and Aadhaar or Emirates ID numbers. What should you do before training?

1. A.Keep it, so the model learns real formats
2. B.Encrypt the files and train on them as is
3. C.Only remove the names of customers
4. D.Detect and mask personal data in the chats

14. File format for hosted fine-tuning

Hosted fine-tuning APIs for chat models usually ask you to upload training data in which format?

1. A.One large CSV with a single text column
2. B.A ZIP of PDF documents
3. C.JSONL, one chat example with messages per line
4. D.A SQL dump of your database

15. Making runs repeatable

Two runs with identical settings give noticeably different validation scores. What is a first step towards repeatable experiments?

1. A.Use a larger tokenizer vocabulary for both
2. B.Set fixed random seeds for data order and init
3. C.Turn off the validation set
4. D.Train on CPU instead of GPU to avoid noise

16. Small data for a narrow task

A team asks if 300 high-quality examples are enough to fine-tune a model to always reply in a fixed JSON shape. What is the best answer?

1. A.Often yes for a narrow format task; test to confirm
2. B.No, fine-tuning needs at least 1 million examples
3. C.Yes, and no test set is needed
4. D.No, JSON can only be learned in pretraining

17. What to record for each adapter

Your team ships LoRA adapters to production every few weeks. Along with each adapter file, what must be recorded so it can be reproduced and served safely?

1. A.The exact base model, data version and training config
2. B.Only the date and the name of the engineer
3. C.Only the final training loss value
4. D.Only the GPU type used for training

18. Early stopping patience

An early-stopping callback has patience=3 and checks validation loss every 200 steps. When does training stop?

1. A.After exactly 3 epochs in total
2. B.After 3 checks in a row with no improvement
3. C.After the validation loss rises by 3%
4. D.After 3 steps in a row with a falling loss

19. What BitFit trains

BitFit is a very light parameter-efficient method. Which parameters does it train?

1. A.Only the attention weights
2. B.Only the embedding table
3. C.Only the bias terms
4. D.Only the final layer norm

20. Centring a long paragraph

Why is centring a long paragraph of body text usually a bad idea?

1. A.The eye must hunt for each line's start
2. B.Centred text cannot be printed in CMYK
3. C.Centred text always uses more ink
4. D.Centred text cannot be translated later

21. Trainable parameters in a LoRA adapter

You add a LoRA adapter of rank 8 to one 4096 x 4096 weight matrix. How many trainable parameters does the adapter add?

1. A.8,192
2. B.32,768
3. C.65,536
4. D.16,777,216

22. Choosing LoRA target modules

Your LoRA adapter only targets q_proj and v_proj, and quality is below full fine-tuning. Which change most often closes the gap?

1. A.Lower the rank from 16 to 2
2. B.Freeze the adapter and train the embeddings
3. C.Remove the validation set to train on more data
4. D.Also adapt the other attention and MLP projections

23. Model never stops after SFT

You set pad_token = eos_token and mask all pad positions with -100. After SFT, the model never stops generating. Why?

1. A.Padding makes the context window too short
2. B.EOS tokens are not allowed in training data
3. C.The learning rate was too low for the EOS row to change
4. D.The real EOS was masked too, so stopping was never learned

24. Quality versus quantity of SFT data

A team has 200,000 auto-scraped Q&A pairs of mixed quality, or could hand-check 2,000 high-quality ones. For teaching a chat style, what does experience (e.g. the LIMA study) suggest?

1. A.Only the largest dataset can change model behaviour
2. B.Mixed-quality data is always better than clean data
3. C.A small, clean, varied set often works as well or better
4. D.Data quality has no effect after 1,000 examples

25. Effect of beta in DPO

In DPO, what happens when you increase beta (for example from 0.1 to 0.5)?

1. A.The policy is allowed to drift further from it
2. B.The rejected responses are ignored
3. C.The loss stops depending on the reference
4. D.The policy stays closer to the reference model

26. What makes ORPO different

ORPO (odds ratio preference optimisation) is often described as a one-stage method. Why?

1. A.It needs no preference data, only prompts
2. B.It adds a preference term to the SFT loss, with no reference model
3. C.It trains the reward model and policy as one network
4. D.It runs RL with PPO in a single epoch

27. Training on thumbs up and down

Your app logs thumbs-up or thumbs-down on single replies, not pairs. Which method is designed for this unpaired binary feedback?

1. A.Classic DPO
2. B.KTO
3. C.Bradley-Terry reward modelling
4. D.Sequence-level distillation

28. Distillation loss on logits

When you have the teacher's logits, which loss is the usual way to make the student match the teacher's softened probabilities?

1. A.Mean squared error on the token IDs
2. B.KL divergence between teacher and student distributions
3. C.Hinge loss on the top-1 class
4. D.Cosine distance between the input embeddings

29. Why on-policy distillation helps

In on-policy distillation, the student generates its own answers and the teacher scores each token. What problem does this address compared with training only on teacher-written text?

1. A.The teacher no longer needs to be loaded at all
2. B.It removes the need for a tokenizer
3. C.The student learns to recover from its own mistakes
4. D.It makes the student larger than the teacher

30. Best-of-n then fine-tune

A team samples 8 answers per prompt, keeps the one with the highest reward score, and runs SFT on those. What is this method called?

1. A.Rejection sampling fine-tuning
2. B.Prompt tuning
3. C.Contrastive pretraining on pairs
4. D.Weight pruning

31. Building preference pairs

A data engineer builds DPO pairs by taking a good answer from one prompt and a bad answer from a different prompt. What is wrong?

1. A.DPO needs three answers per pair
2. B.Both answers must respond to the same prompt
3. C.The rejected answer must be empty
4. D.Nothing, any two answers can be paired

32. Classifier ignores the rare class

Your fine-tuned classifier has 97% accuracy but catches almost no fraud cases, which are 3% of data. Which change targets this directly?

1. A.Train more epochs with the same loss
2. B.Report accuracy only on the majority class
3. C.Lower the learning rate by ten times
4. D.Use class-weighted loss or oversample fraud rows

33. Choosing a metric for free text

Your fine-tuned model writes short explanations of bank charges. Why is exact-match accuracy a poor metric here?

1. A.Many different wordings can be equally correct
2. B.Exact match is too slow to compute
3. C.Exact match only works for images
4. D.It rewards answers that are too short

34. Training a model to call tools

To fine-tune a model to use tools, what should the training conversations include?

1. A.Only the final reply with tools removed
2. B.Only the tool schema with no examples
3. C.Tool calls, their results and the final reply
4. D.Random tool calls with no results shown

35. Time for a self-hosted run

Your setup trains at 3,000 tokens per second. The dataset is 18 million tokens and you train for 2 epochs. Roughly how long will training take?

1. A.About 1.7 hours
2. B.About 6.7 hours
3. C.About 3.3 hours
4. D.About 33 hours

36. Long examples cut at max length

You set max_seq_length=1024, but 20% of examples are 2,000 tokens long with the answer at the end. What happens with right-side truncation?

1. A.Those answers are cut off, so the model learns little from them
2. B.The model automatically summarises the long prompts
3. C.The examples are split into two full examples
4. D.Training fails with an error on long rows

37. OOM after doubling sequence length

A LoRA run fits at max length 2,048 but runs out of memory at 4,096 with the same batch size. Which part of memory grew most?

1. A.The frozen base model weights in 4-bit
2. B.The LoRA adapter weights and their state
3. C.The tokenizer vocabulary
4. D.Stored activations for the backward pass

38. Order of domain training steps

You have 1 GB of raw insurance documents and 3,000 labelled Q&A pairs. What is a sensible order of training?

1. A.SFT on Q&A, then DPO on the raw documents
2. B.Only SFT on the documents as plain answers
3. C.Continued pretraining on documents, then SFT on Q&A
4. D.Reward modelling on documents, then pretraining

39. Big price next to small text

In a pricing row, a 40 px price sits next to a 14 px "/month" label, both centred vertically, and it looks off. What alignment usually looks best?

1. A.Align both on a shared baseline
2. B.Align their tops to the same line
3. C.Centre them on the row's height
4. D.Align the small text to the price's descender

40. An opening quote that dents the margin

A large pull quote starts with a curly opening quote mark, and the first line looks indented compared with the lines below. What is the professional fix?

1. A.Hang the quote mark outside the text edge
2. B.Increase tracking on the first line
3. C.Delete the opening quote mark
4. D.Justify the whole quote

41. How DPO differs from PPO-based RLHF

How does Direct Preference Optimization (DPO) differ from classic PPO-based RLHF?

1. A.It needs no preference data of any kind
2. B.It only updates the tokenizer and embeddings
3. C.It needs a larger reward model than PPO does
4. D.It trains on preference pairs with no separate reward model or RL loop

42. Merging a QLoRA adapter

You trained a QLoRA adapter on a 4-bit base and want one merged model for serving. Which approach keeps quality best?

1. A.Merge directly into the 4-bit weights and save
2. B.Average the adapter with a random adapter
3. C.Retrain the adapter in 4-bit after merging
4. D.Load the base in fp16, merge, then quantise if needed

43. Counting optimizer steps

You have 12,000 training examples, train for 3 epochs, with 2 GPUs, per-device batch size 4 and gradient accumulation 4. About how many optimizer steps will run?

1. A.2,250
2. B.9,000
3. C.563
4. D.1,125

44. Loss on multi-turn chats

Your SFT data has multi-turn chats (system, user, assistant, user, assistant). A good default is to compute loss on:

1. A.Every assistant turn, with system and user masked
2. B.Only the system prompt of each chat
3. C.Every token, including all the user turns
4. D.Only the first user message

45. NaN loss in fp16 fine-tuning

Full fine-tuning in fp16 on an A100 hits NaN loss after a few hundred steps, even with a loss scaler. Which change most often fixes it on this hardware?

1. A.Switch to fp8 for both weights and gradients
2. B.Train in bf16, which has a much wider exponent range
3. C.Raise the learning rate so updates outrun overflow
4. D.Turn off gradient clipping entirely

46. Checkpointing with frozen inputs

You turn on gradient checkpointing for a LoRA run where the embedding layer is frozen, and the adapters get no gradient. What is the usual fix in Hugging Face?

1. A.Turn off the LoRA dropout setting
2. B.Set the learning rate to zero for embeddings
3. C.Use fp32 for the frozen base weights
4. D.Call enable_input_require_grads() on the model

47. Weight decay on LoRA

What does decoupled weight decay (AdamW) do to LoRA adapter parameters during training?

1. A.Pulls the adapter, and so the model, towards the base
2. B.Pulls the base model weights towards zero
3. C.Has no effect, because the adapters start at zero
4. D.Increases the adapter rank over time

48. Chosen likelihood falls during DPO

During DPO, the margin between chosen and rejected keeps growing, but the log probability of chosen answers also falls steadily. Which change is often used to counter this?

1. A.Set beta to zero to remove the reference
2. B.Add an SFT (NLL) term on the chosen answers
3. C.Swap chosen and rejected in half the pairs
4. D.Train the reward model for more epochs

49. Off-policy preference data

You run DPO on your 8B model using public preference pairs whose answers were written by very different models. Gains are small. What usually helps most?

1. A.Raise beta until the loss falls to zero
2. B.Train on the rejected answers with SFT
3. C.Build pairs from your own model's samples, then label them
4. D.Use pairs from the other models' prompts only

50. Advantages in a GRPO group

In GRPO, four sampled answers to one maths prompt get rewards [1, 0, 0, 1]. Using (reward − mean) / std with population std, what are their advantages?

1. A.[+0.5, −0.5, −0.5, +0.5]
2. B.[1, 0, 0, 1]
3. C.[+2, −2, −2, +2]
4. D.[+1, −1, −1, +1]

51. Process and outcome reward models

How does a process reward model (PRM) differ from an outcome reward model (ORM) for multi-step reasoning?

1. A.A PRM is trained only on images and audio
2. B.A PRM scores each step; an ORM only the final result
3. C.An ORM needs human labels for every token
4. D.They are just two names for the same model

52. Merging two fine-tunes of one base

You have two full fine-tunes of the same base model, one for Arabic and one for SQL. What does "task arithmetic" merging do?

1. A.Adds both weight deltas from the base to the base
2. B.Averages the two tokenizers into a new one
3. C.Runs both models and picks the longer answer
4. D.Trains a third model from scratch on both datasets

53. Averaging several fine-tuned runs

You fine-tuned the same pretrained model 5 times with different hyperparameters. Averaging their weights (a "model soup") often works. Why?

1. A.Averaging always works for any two models at all
2. B.It removes the need for a test set
3. C.It makes the merged model five times larger
4. D.Runs from one start tend to share a low-loss basin

54. Fine-tuning with differential privacy

DP-SGD gives formal privacy guarantees for training on sensitive records. What does it do on each step?

1. A.Encrypts the dataset before each epoch
2. B.Removes rare words from the vocabulary
3. C.Clips each example's gradient and adds noise
4. D.Trains on a random 1% of the examples only

55. Rank sweep results

Validation scores for LoRA ranks 8, 16, 32, 64 are 71.2, 74.8, 75.1 and 74.9. Serving many adapters makes size matter. Which rank is the best choice?

1. A.16, since higher ranks add size for no clear gain
2. B.64, since the largest rank is always safest
3. C.8, since the smallest rank always generalises best
4. D.32, since it has the single highest score

56. Splitting chat logs for evaluation

Your dataset has 40,000 chats from 2,000 customers, many repeating similar issues. You split rows randomly into train and test, and the test score is very high. What is the better split?

1. A.Split by row length so long chats are in test
2. B.Split by customer so no customer is in both sets
3. C.Use the training set as the test set
4. D.Split randomly but with a larger test set

57. Fine-tuned small model versus API

A large API model costs ₹0.40 per request. A fine-tuned 8B model on a rented GPU costs ₹150 per hour and handles 2,000 requests an hour at full load. At full load, what is the cost per request on the GPU?

1. A.₹0.75
2. B.₹0.075
3. C.₹0.30
4. D.₹0.0075

58. Offline win, online question

Your new fine-tune beats the old one on offline evals. Before full rollout to a UAE bank's customer chat, what is the best way to confirm it helps real users?

1. A.Switch all traffic at once and watch for complaints
2. B.Re-run the offline evals with a larger batch size
3. C.Ask the model to rate its own conversations
4. D.Send a small share of traffic to it and compare business metrics

59. Fine-tuned model scores too well

Your fine-tuned support model scores 97% on the eval set but users report many failures. You find the training data was built from the same ticket export as the eval set. What is the most likely problem?

1. A.The learning rate was too low, so the model never fully converged
2. B.The LoRA rank was too high, so the adapter ignored the base model
3. C.Near-duplicates leaked across train and eval, inflating the score
4. D.The eval set was run at temperature 0 instead of production settings

60. When prompting is not enough

A team has tried careful prompts and 20 examples for a niche medical coding task. Accuracy plateaued at 78%, and they have 30,000 labelled examples. What is a reasonable next step?

1. A.Add all 30,000 examples to every prompt
2. B.Consider fine-tuning on the labelled data and compare on a held-out set
3. C.Keep rewording the instructions until accuracy improves
4. D.Switch the temperature to 1 for more creative codes