Design a multi-region disaster recovery setup for a payments API with an RPO of under 1 minute and an RTO of 15 minutes. What trade-offs do you consider?
What the interviewer wants to hear
- Translate RTO/RPO into a strategy such as warm standby
- Asynchronous cross-region data replication and its lag
- Traffic failover with DNS health checks or global load balancing
- Infrastructure as code, replicated secrets, keys and images in the second region
- Regular failover drills, runbooks and clear failover decision
- Cost, consistency and split-brain risks
Model answer
An RPO under a minute rules out periodic backups, and an RTO of 15 minutes rules out building a region from scratch, so I would use warm standby. The primary region runs the full stack. The secondary runs a smaller copy deployed from the same Terraform and pipeline. Data uses Aurora Global Database or DynamoDB global tables, which replicate asynchronously with typical lag around a second, meeting the RPO. Secrets, KMS keys, container images and config are replicated too. Route 53 failover records or a global load balancer shift traffic, but I prefer a human-approved or well-tested automated failover to avoid flapping. Key trade-offs: async replication means a few seconds of transactions can be lost, so payments need idempotency keys and reconciliation; split brain must be prevented by fencing the old primary. Standby costs money, and the plan only works if we run regular game days.
Can you answer it under time pressure?
Take a timed hard round and get scored on every answer.
More hard Cloud Engineer questions
- How would you design a multi-account cloud landing zone for a company with 20 product teams?
- Leadership asks you to reduce the cloud bill by 30 percent without hurting reliability. How do you approach it?
- How would you migrate a large production database to a managed cloud database with near-zero downtime?
- A company needs reliable connectivity between its data center and the cloud. Compare site-to-site VPN and a dedicated connection like AWS Direct Connect, and describe a resilient design.
- Design a shared Kubernetes platform used by 30 product teams. How do you handle isolation, security, cost and upgrades?
- You find out that long-lived AWS access keys were pushed to a public GitHub repository an hour ago. How do you respond?
- Design observability and alerting for a customer-facing API so on-call gets paged for real problems but not for noise.
- Design an event-driven order processing system on a public cloud that must handle traffic spikes and never lose or double-charge an order.
- Your company has hundreds of Terraform configurations owned by many teams. How do you structure modules, state and governance so it scales safely?