Your team is burning out from on-call pages. How would you redesign alerting and the on-call process?
What the interviewer wants to hear
- Audit alerts: delete or demote ones with no action
- Page only on user-facing symptoms and SLO burn rates
- Every page links to a runbook and dashboard
- Route by severity: page, ticket or dashboard
- Track page volume and review in regular meetings
- Fix recurring causes with automation and follow-up work
Model answer
I would start with data: export the last month of pages and tag each one as actionable, duplicate or noise. Alerts that never needed human action get deleted or turned into tickets or dashboard panels. For paging I would move to symptom-based alerts on user impact, mainly multi-window SLO burn-rate alerts for availability and latency, instead of CPU or single-host checks. Every remaining page must have a clear owner, a severity, and a link to a runbook and dashboard. I would group related alerts in Alertmanager so one incident causes one page, and add inhibition so downstream alerts are muted when an upstream one fires. On the process side, a weekly on-call review looks at page counts, sleep-hour pages and recurring causes, and turns them into backlog items with real priority. We also make sure rotations are fair and that time is reserved after a heavy shift.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium DevOps Engineer questions
- How do you manage secrets such as database passwords and API keys across CI/CD pipelines and Kubernetes?
- A Pod is stuck in CrashLoopBackOff after a deployment. How do you troubleshoot it?
- Compare rolling, blue-green and canary deployments. How do you decide which to use?
- Developers complain that the CI pipeline takes 40 minutes. How would you make it faster?
- How do you structure Terraform code for multiple environments like dev, staging and production?
- When would you package Kubernetes manifests with Helm and when with Kustomize? Can they be combined?
- Users report that one service on Kubernetes became slow since this morning. Walk me through how you would investigate.
- How would you promote a release from dev to staging to production in a GitOps setup with Argo CD?
- How do you include database schema migrations in a continuous delivery pipeline without causing downtime?