Latency
15 practice items · needed by 3 roles
Easy 1Medium 5Hard 9
Practice questions
- Percentiles vs average latencymedium
- p99 latency across all instancesmedium
- Combining percentiles from many servershard
- Tail latency with fan-outhard
- Coordinated omissionhard
- Misleading histogram quantileshard
- Making prompt caching workmedium
- What streaming improveshard
- Where generation time goeshard
- Estimating response timehard
- How speculative decoding helpshard
- What p99 latency meanseasy
- Concurrent requests at peakmedium
- Fleet-wide p99 from many podsmedium
- Tail latency at low CPU usagehard