MLOps Engineer
Performance and debugging questions
53 questions on Performance and debugging for mlops engineers, easiest first.
Easy 8
Medium 20
- Concurrent requests at peakmedium
- Fleet-wide p99 from many podsmedium
- Timing a GPU operationmedium
- Memory coalescingmedium
- If-else inside a warpmedium
- Low occupancy in a kernelmedium
- Why GELU and adds are memory-boundmedium
- Odd layer sizes run slowlymedium
- Finding the slow part of a stepmedium
- Gaps in the GPU timelinemedium
- Logging slows the training loopmedium
- How DDP hides communicationmedium
- Training hangs with no errormedium
- Scaling efficiency on 8 GPUsmedium
- Slow first CUDA call on bare metalmedium
- Reading a PromQL latency querymedium
- Setting a latency SLOmedium
- GPU idle while CPU is maxedmedium
- Cross-encoder reranker is slowmedium
- Small model leaves the GPU idlemedium
Hard 25
- Tail latency at low CPU usagehard
- More threads, no extra throughputhard
- Arithmetic intensity of c = a + bhard
- Arithmetic intensity of a big matmulhard
- Time for one elementwise passhard
- Time for a 4K matmulhard
- Computing MFU for a training runhard
- One GPU is slower than its peershard
- Error points to the wrong linehard
- Device-side assert in cross-entropyhard
- Overlapping host-to-device copieshard
- Time to all-reduce 7B gradientshard
- Multi-node job runs at PCIe speedhard
- One node's link runs at half speedhard
- Data prep and compute in sequencehard
- Bug appears only on some nodeshard
- Combining quantiles across servershard
- Upper bound on single-stream speedhard
- Why batching raises decode throughputhard
- Estimating prefill timehard
- Where 4-bit weights speed things uphard
- Slow first token for Dubai usershard
- When decode stops being memory-boundhard
- Why a 100K prompt is so slowhard
- Latency explodes near full loadhard