One slow downstream dependency caused your whole platform to go down. How would you handle the incident, and what would you change afterwards?

What the interviewer wants to hear

• Mitigate first: shed load, disable the feature, roll back, scale
• Explain the mechanism: threads or connections exhausted by waiting on slow calls
• Timeouts with deadlines, circuit breakers and bulkheads
• Limit retries with backoff, jitter and retry budgets
• Load shedding and graceful degradation
• Blameless postmortem with action items and better alerting

Model answer

During the incident I focus on stopping the damage, not on the root cause: turn off the feature that calls the dependency with a flag, shed non-critical traffic, or roll back if a deploy triggered it, while keeping a timeline in the incident channel. The usual mechanism is that callers wait on the slow service with long timeouts, so their threads and connection pools fill up, and then their callers do the same. Retries at every layer multiply the load. Afterwards I would add short timeouts with propagated deadlines, circuit breakers that fail fast, and bulkheads, meaning separate pools per dependency so one cannot starve the others. Retries would be limited to one layer, with backoff, jitter and a retry budget. I would add load shedding and degraded responses, such as cached data. Finally, a blameless postmortem with owned action items and alerts on dependency latency.

More hard Backend Developer questions

• Design a URL shortener like bit.ly that handles 100 million new links per month and many more redirects. Cover IDs, storage, caching and scale.
• A product manager says each payment event must be processed exactly once. How do you achieve that with real message queues and networks?
• Your single PostgreSQL database is reaching its limits. How would you scale reads and then writes, and what problems does each step bring?
• In a microservices system, placing an order touches the Order, Payment and Inventory services, each with its own database. How do you keep data consistent?
• You are building a B2B SaaS backend used by many companies. How would you design multi-tenancy and make sure one tenant can never see another tenant data?
• Design a notification service that sends email, SMS and push messages for many products, up to tens of millions of messages a day.
• Your company wants to split a large monolith into microservices. How would you approach it, and when would you argue against it?
• Design the backend for a real-time chat app with one-to-one and group chats, online status and message history.
• How would you design a ledger for a wallet or payments product so balances are always correct?