Site Reliability Engineer
Observability and monitoring questions
74 questions on Observability and monitoring for site reliability engineers, easiest first.
Easy 23
- The four golden signalseasy
- How Prometheus collects metricseasy
- The USE methodeasy
- The RED methodeasy
- Counter or gauge?easy
- Why structured logs?easy
- Spans in a traceeasy
- Grouping in Alertmanagereasy
- Grafana template variableseasy
- How Loki indexes logseasy
- A server with almost no free RAMeasy
- Re-running a command every few secondseasy
- What increase() returnseasy
- The up metriceasy
- What node_exporter provideseasy
- Naming a new metriceasy
- Finding the busiest endpointseasy
- Quieting alerts during maintenanceeasy
- Links inside alertseasy
- What OTLP iseasy
- Tracing with little codeeasy
- Log level in productioneasy
- Observability vs monitoringeasy
Medium 29
- Using rate() on a countermedium
- Which alert should page?medium
- Percentiles vs average latencymedium
- p99 latency across all instancesmedium
- Alerting when a job disappearsmedium
- The for clause in alert rulesmedium
- Inhibition in Alertmanagermedium
- A label that broke Prometheusmedium
- Why recording rules?medium
- Why run an OpenTelemetry Collector?medium
- Traces broken between servicesmedium
- Counting errors with LogQLmedium
- Reading a slow tracemedium
- Catching certificate expirymedium
- CPU usage at 3 AM yesterdaymedium
- irate or rate?medium
- Metrics from a nightly jobmedium
- Checking a site from outsidemedium
- What a ServiceMonitor doesmedium
- Aggregating with without()medium
- Comparing with last weekmedium
- Alerting before a disk fillsmedium
- Jumping from a graph to a tracemedium
- The traceparent headermedium
- Why errors go missing from tracesmedium
- Joining logs and tracesmedium
- Calculating Apdexmedium
- kube-state-metrics or cAdvisormedium
- Counting time seriesmedium
Hard 22
- How much budget does a burn rate use?hard
- Combining percentiles from many servershard
- Why a short window in burn-rate alertshard
- Gaps in a rate() graphhard
- Misleading histogram quantileshard
- Double pages from HA Prometheushard
- Broken tail-based samplinghard
- Alerting on queue laghard
- SLOs for a low-traffic servicehard
- Where to measure the SLIhard
- Reading /proc/pressure/memoryhard
- A 10-minute scrape intervalhard
- Pushgateway hiding a dead jobhard
- Dropping a noisy metrichard
- Adding a label from another metrichard
- Fleet error rate done righthard
- Reading histogram bucketshard
- Bucket edges and the SLOhard
- One span, many parent traceshard
- An alert reaches only one teamhard
- Collector crashing under burstshard
- A child span before its parenthard