Hard Machine Learning Engineer questions

32 questions · 20 points each for a correct first answer.

1. 1.Great offline, bad in productiontraining-serving-skew, feature-storehard
2. 2.Why fp16 needs loss scaling but bf16 usually does notmixed-precision, deep-learninghard
3. 3.Model does not fit in GPU memorydistributed-training, gpu-memoryhard
4. 4.ROC-AUC after calibrationcalibration, classification-metricshard
5. 5.Better offline metric, worse A/B testranking, ab-testinghard
6. 6.L2 penalty versus AdamWoptimisation, regularisationhard
7. 7.A feature that is too gooddata-leakage, feature-engineeringhard
8. 8.Tuning an HNSW vector indexvector-search, model-servinghard
9. 9.KV cache size for one sequencekv-cache, llm-servinghard
10. 10.Parameters in a multi-head attention blocktransformers, attentionhard
11. 11.view() after a transposepytorch, tensor-shapeshard
12. 12.Fine-tuning a CNN with batch size 2normalization, fine-tuninghard
13. 13.Why Adam divides by 1 - beta^toptimizershard
14. 14.What ZeRO stage 3 shardsdistributed-traininghard
15. 15.Activation checkpointing trade-offdistributed-training, memory-optimizationhard
16. 16.Why RoPE encodes relative positiontransformers, positional-encodinghard
17. 17.What FlashAttention changesattention, gpu-performancehard
18. 18.Speculative decoding and output qualityllm-serving, inference-optimizationhard
19. 19.Why serving engines page the KV cachellm-serving, kv-cachehard
20. 20.INT8 activations hurt accuracyquantizationhard
21. 21.Why scale distillation loss by T squareddistillationhard
22. 22.How DPO differs from PPO-based RLHFalignment, fine-tuninghard
23. 23.Fine-tuned model forgets general skillsfine-tuning, llmhard
24. 24.Filtered vector search returns too fewvector-search, raghard
25. 25.Inputs stable but accuracy fallsmonitoring, drifthard
26. 26.Retraining on your own recommendationsrecommender-systems, biashard
27. 27.Random ID tops feature importanceinterpretability, random-foresthard
28. 28.SMOTE applied before cross-validationdata-leakage, class-imbalancehard
29. 29.LLM judge prefers the first answerllm-evaluationhard
30. 30.Instructions hidden in a retrieved pagellm-safety, agentshard
31. 31.Daily cost of an LLM featurecost-optimization, llm-servinghard
32. 32.Negative R-squared on test dataregression-metricshard