Skip to content
RED SAGSkills
mediumAutomation Test Engineer interview question

A test fails randomly in CI about 1 in 20 runs. Walk me through how you find and fix the cause.

What the interviewer wants to hear

  • Reproduce by running it many times, locally and in CI conditions
  • Use traces, screenshots and logs from failures
  • Check common causes: timing, shared data, test order, environment, animations
  • Fix the root cause rather than adding sleeps or retries
  • Quarantine meanwhile and track flakiness

Model answer

First I try to reproduce it: run the test 50 or 100 times, for example with Playwright's --repeat-each, with the same workers and CPU limits as CI. Then I study failure artifacts: the trace, screenshot, console and network logs. Usually the cause is one of a few things. Timing: the test acts before data loads, or uses a fixed sleep, so I switch to waiting for a specific condition or web-first assertion. Shared data: another test changes the same user, so I make data unique per test. Order dependence: the test only passes after another test ran. Environment: a slow third-party service, so I mock it. Animations or toasts covering a button. Sometimes the flakiness is a real app race condition, and then I log a bug. While I work on it, I quarantine the test so it does not block others, and I track the fix.

Can you answer it under time pressure?

Take a timed medium round and get scored on every answer.

Start mock interview

More medium Automation Test Engineer questions