You find out that long-lived AWS access keys were pushed to a public GitHub repository an hour ago. How do you respond?
What the interviewer wants to hear
- Immediately deactivate the key and revoke any active sessions
- Investigate CloudTrail for actions taken with that key across all regions
- Look for persistence: new users, keys, roles, Lambda functions, instances
- Contain and clean up, check billing for crypto mining
- Remove from history but treat as compromised regardless
- Root cause and prevention: OIDC, no long-lived keys, secret scanning
Model answer
I treat it as an active incident, because bots scan GitHub for keys within minutes. First I deactivate the access key, and if it belonged to a user with assumed role sessions I revoke those sessions with a deny policy on tokens issued before now. Then I investigate CloudTrail for every API call made with that key ID in all regions, and check GuardDuty findings. I look for persistence such as new IAM users, access keys, roles with odd trust policies, Lambda functions, EC2 instances in unused regions and changes to logging, plus a billing spike from crypto mining. I remove anything the attacker created and rotate any secrets they could have read. Rewriting Git history is worth doing, but the key stays compromised either way. Afterwards I run a postmortem and remove long-lived keys entirely in favor of OIDC and roles, with secret scanning and push protection enabled.
Can you answer it under time pressure?
Take a timed hard round and get scored on every answer.
More hard Cloud Engineer questions
- Design a multi-region disaster recovery setup for a payments API with an RPO of under 1 minute and an RTO of 15 minutes. What trade-offs do you consider?
- How would you design a multi-account cloud landing zone for a company with 20 product teams?
- Leadership asks you to reduce the cloud bill by 30 percent without hurting reliability. How do you approach it?
- How would you migrate a large production database to a managed cloud database with near-zero downtime?
- A company needs reliable connectivity between its data center and the cloud. Compare site-to-site VPN and a dedicated connection like AWS Direct Connect, and describe a resilient design.
- Design a shared Kubernetes platform used by 30 product teams. How do you handle isolation, security, cost and upgrades?
- Design observability and alerting for a customer-facing API so on-call gets paged for real problems but not for noise.
- Design an event-driven order processing system on a public cloud that must handle traffic spikes and never lose or double-charge an order.
- Your company has hundreds of Terraform configurations owned by many teams. How do you structure modules, state and governance so it scales safely?