How would you design a test automation framework from scratch for a web app?
What the interviewer wants to hear
- Choose language and tools that match the team and app
- Clear structure: tests, page or component objects, fixtures, utilities, test data
- Configuration for environments and secrets
- Reporting, logging, screenshots and traces
- Parallel execution and CI integration
- Coding standards and code review
Model answer
I start with the context: the app's stack, the team's language skills and the browsers we must support. For a modern web app with TypeScript developers, I would pick Playwright Test, so developers can contribute too. The structure has folders for tests grouped by feature, page or component objects, fixtures for setup like authenticated users, API clients for data setup, and test data factories that create unique data. Configuration handles base URLs per environment through environment variables, with secrets kept out of the code. I add tagging for smoke and regression, retries only in CI and with flakiness tracking, and reports with traces, screenshots and videos on failure. Tests run in parallel and sharded in GitHub Actions. I add linting, including no-floating-promises, and code review rules. Finally I document how to write a new test so the framework is easy to adopt.
Can you answer it under time pressure?
Take a timed medium round and get scored on every answer.
More medium Automation Test Engineer questions
- A test fails randomly in CI about 1 in 20 runs. Walk me through how you find and fix the cause.
- How do you manage test data in an automated suite that runs in parallel on shared environments?
- When should automated tests mock external services, and when should they use real ones?
- How do you make automated test results useful to developers and managers?
- How do you handle authentication in a large UI test suite without making every test log in through the UI?
- Your suite needs to run in parallel across many workers. What do you change to make that safe?
- Describe how you would set up automated tests in a CI/CD pipeline for a web application.
- You are asked to load test a new checkout API before a big sale. How do you plan and run it?
- How would you approach test automation for a mobile app on Android and iOS?