Explain how an A/B test works and when you would use one.
What the interviewer wants to hear
- Random split between control and variant
- Hypothesis and one primary metric defined up front
- Enough sample size and duration
- Guardrail metrics to catch harm
- Not suitable for low traffic or big strategic bets
Model answer
In an A/B test, users are randomly split into a control group that keeps the current experience and a variant group that gets the change. Because the split is random, a difference in results can be attributed to the change, not to seasonality or marketing. Before starting, I write a hypothesis, choose one primary metric such as checkout conversion, add guardrail metrics like refunds or page speed, and calculate how many users and days I need to detect a meaningful effect. Then I let it run for the planned time, usually whole weeks, without stopping early when it looks good. I use A/B tests when there is enough traffic and the change is measurable, like copy, flows or pricing pages. For low-traffic products or big strategic bets, I rely more on qualitative research and staged rollouts.
Can you answer it under time pressure?
Take a timed easy round and get scored on every answer.
More easy Product Manager questions
- In your own words, what does a product manager do?
- How do you decide which features to build first?
- What is an MVP? Give an example of how you would build one.
- How do you work with engineers and designers?
- Pick a product you use often. How would you improve it?
- How do you write user stories and acceptance criteria that your team can build from?
- What is the product manager's role in agile ceremonies like planning, stand-ups, reviews and retrospectives?
- What makes a good product roadmap?
- Customers send feedback through support, sales, reviews and surveys. How do you use it?