Design CI/CD for a monorepo with 60 services and 200 engineers where full builds take over an hour.
What the interviewer wants to hear
- Build only what changed using a dependency graph of affected targets
- Remote caching and remote execution with tools like Bazel, Nx or Turborepo
- Parallelize on autoscaled ephemeral runners
- One aggregate required check per PR; merge queue for main
- Independent deploys per service with ownership via CODEOWNERS
- Measure pipeline time and flakiness and fix flaky tests
Model answer
The core idea is to never build the whole repo for a normal change. I would adopt a build tool that understands the dependency graph, such as Bazel, Nx, Turborepo or Pants, so CI computes the affected targets from the diff and only builds and tests those and their dependants. A shared remote cache means unchanged targets are fetched instead of rebuilt, across CI and developer laptops. Work runs in parallel on autoscaled ephemeral runners. For merging, branch protection requires one aggregate status check, so path-skipped jobs never block a PR, and a merge queue tests changes against the latest main to keep it green. Each service still deploys independently: merging to main builds its image, and GitOps promotes it per environment, with CODEOWNERS routing reviews. I would track p50 and p95 pipeline time, cache hit rate and flaky test rate on a dashboard, quarantine flaky tests quickly, and run a full build nightly as a safety net.
Can you answer it under time pressure?
Take a timed hard round and get scored on every answer.
More hard DevOps Engineer questions
- You are asked to build an internal platform so 30 product teams can deploy services to Kubernetes on their own. How would you design it?
- How would you secure the software supply chain from source code to a running container in production?
- Your company wants to run a critical service active-active in two cloud regions. What are the main DevOps challenges and how would you handle them?
- Your DORA metrics show a 30% change failure rate and long recovery times. How would you bring them down without slowing teams down?
- How do you plan and carry out a Kubernetes version upgrade for production clusters with minimal risk?
- Leadership asks you to cut the cloud bill for your Kubernetes platform by 30 percent without hurting reliability. How do you approach it?
- You are the incident commander for a major outage affecting all customers. Describe how you run the incident from first page to postmortem.
- Design a disaster recovery plan for a platform whose business requires an RTO of 1 hour and an RPO of 5 minutes for the loss of a whole cloud region.
- Several teams will share one Kubernetes cluster. How would you isolate tenants and secure the cluster?