How would you add rate limiting to a public API that runs on many servers?

What the interviewer wants to hear

• Decide what to limit by: API key, user, IP, and per endpoint cost
• Choose an algorithm such as token bucket or sliding window
• Shared state in Redis with atomic operations or Lua scripts
• Return 429 with Retry-After and rate limit headers
• Fail open or closed decision, and limits at the edge or gateway

Model answer

First I would decide the key: usually the API key or user id, with IP limits for anonymous traffic, and stricter limits on expensive endpoints like search or login. For the algorithm I prefer a token bucket, which allows short bursts but enforces an average rate, or a sliding window counter, which avoids the double burst at fixed window edges. Because there are many servers, the counters must be shared, so I would keep them in Redis and update them atomically with a Lua script or INCR plus EXPIRE. When a client is over the limit I return 429 with a Retry-After header, and on every response headers showing the limit and remaining quota. I would also decide what happens if Redis is down; for most APIs I fail open with a local in-memory limit. Coarse limits can also live at the API gateway or CDN to stop floods early.

More medium Backend Developer questions

• An API endpoint is slow because it runs a heavy database query. How would you add caching, and how do you keep the cache correct?
• You need to rename a column in a large production table that the live app uses. How do you do it without downtime?
• How do you change a public API without breaking existing clients, such as old mobile app versions?
• Users say one API endpoint has become slow in production. How do you find and fix the cause?
• Sending emails and generating PDF invoices makes your checkout API slow. How would you move this work to the background?
• You are asked to review the security of a REST API before launch. What would you check?
• Your service receives webhooks from a payment provider. How do you make handling them secure and reliable?
• When would you choose Kafka and when RabbitMQ for messaging between services?
• How would you design an API that lets users upload large files, such as videos of several gigabytes?