How would you measure whether a SOC is effective? Which metrics would you use, and what are their pitfalls?
What the interviewer wants to hear
- Time metrics: MTTD, MTTR, time to contain
- Quality metrics: true positive rate, escalation accuracy
- Coverage: ATT&CK coverage tested by emulation
- Pitfalls: gaming volume metrics, speed over quality
- Tie metrics to business risk and outcomes
Model answer
I would use a mix of speed, quality and coverage metrics. For speed, mean time to detect, acknowledge, contain and recover show how quickly we act, measured from the first malicious event where possible. For quality, I track the true positive rate per detection, how often escalations are confirmed, and repeat incidents. For coverage, I use ATT&CK mapping validated by purple team exercises, not just rules that exist on paper. Each has pitfalls. Counting closed alerts rewards rushing, and pushing MTTR down can cause shallow investigations. Low incident counts may mean poor detection, not safety. So I report metrics as trends with context, review samples of closed tickets for quality, and link results to business risk, such as how quickly we would stop a ransomware attack.
Can you answer it under time pressure?
Take a timed hard round and get scored on every answer.
More hard Cybersecurity Analyst questions
- Ransomware is spreading across your network on a Friday night. Walk me through how you would handle it.
- Design a threat hunt to find lateral movement in a Windows Active Directory environment.
- Your CISO wants to move to zero trust. What does that mean in practice and where would you start?
- A developer accidentally pushed AWS access keys to a public GitHub repository two hours ago. How do you respond?
- How would you build, test and maintain a new detection rule, for example for credential dumping from LSASS?
- Several employees fell for an adversary-in-the-middle phishing site that stole their session cookies despite MFA. How do you respond and harden the environment?
- You are building detection for a cloud-first company with a limited SIEM budget. Which log sources do you onboard first and why?
- HR tells you an engineer who just resigned may be taking source code and customer data. How do you investigate fairly and legally?
- A production Linux server is suspected to be compromised and may lead to legal action. How do you collect evidence in a forensically sound way while limiting business impact?