Skip to content
RED SAGSkills
mediumQA Engineer interview question

Your automated UI suite has become flaky and the team has started ignoring failures. How do you fix the situation?

What the interviewer wants to hear

  • Measure: identify flaky tests and how often they fail, using CI history
  • Quarantine flaky tests so the main pipeline is trusted again, with tickets and owners
  • Find root causes: timing and waits, shared test data, environment, test order
  • Fix with condition-based waits, isolated data, stable locators and mocks where right
  • Prevent: review new tests, track the flaky rate, and do not hide failures with retries

Model answer

First I would rebuild trust in the pipeline. Using CI history I would list tests that fail and then pass on the same code, and how often. Those go into a quarantine group that still runs but does not block merges, each with a ticket and owner. Then I look for root causes using traces, videos and logs. The usual causes are fixed sleeps or missing waits, shared accounts or data between parallel tests, dependence on test order, unstable environments and brittle locators. Fixes are web-first assertions, unique data per test created through the API, role or test-id locators and mocking unreliable third parties. I would also agree a rule that a test leaves quarantine only after many clean runs, review new tests for these patterns and report the flaky rate each sprint. Retries are fine as a buffer, but not as a hiding place.

Can you answer it under time pressure?

Take a timed medium round and get scored on every answer.

Start mock interview

More medium QA Engineer questions