MLOps Engineer
Observability and monitoring questions
36 questions on Observability and monitoring for mlops engineers, easiest first.
Easy 15
- What data drift iseasy
- What p99 latency meanseasy
- Quick check of GPU stateeasy
- GPU metrics in Prometheuseasy
- Service health basicseasy
- Joining predictions to labelseasy
- What a drift runbook giveseasy
- Nobody owns the alerteasy
- Mixing versions in metricseasy
- Too many charts on the dashboardeasy
- Paging for minor drifteasy
- Weekly chart looks calmeasy
- Known requests sent on a timereasy
- Planning monitoring earlyeasy
- Dashboard for an LLM serviceeasy
Medium 11
- Labels arrive weeks latermedium
- What to log for each LLM callmedium
- Fleet-wide p99 from many podsmedium
- Online features stopped updatingmedium
- Catching GPU faults earlymedium
- One bad minute pages on-callmedium
- Sampling logs to save costmedium
- Aadhaar numbers in logsmedium
- Reading a PromQL latency querymedium
- How useful are the alertsmedium
- Accuracy slips months after launchmedium
Hard 10
- Drift alerts firing all the timehard
- Model affects its own training datahard
- GPU shows 100% but is underusedhard
- Xid errors in the kernel loghard
- Chance alarms over 200 featureshard
- Drift on an unimportant featurehard
- user_id as a metric labelhard
- Drift job silently stoppedhard
- Alerts from tiny segmentshard
- Setting alert thresholdshard