Experimentation
20 practice items · needed by 2 roles
Easy 4Medium 7Hard 9
Practice questions
- A/B test versus canaryeasy
- Canary that cannot catch quality dropshard
- What is an MVPeasy
- The control groupeasy
- Writing a hypothesiseasy
- Defining success before launchmedium
- A fake door testmedium
- What a p-value meansmedium
- Detecting a smaller effectmedium
- Guardrail metricsmedium
- Checking results every daymedium
- An uneven 50/50 splitmedium
- A winning test with a hidden costhard
- Leadership wants an AI chatbothard
- Do the wins add uphard
- Twenty metrics, one winnerhard
- Testing in a two-sided marketplacehard
- A positive but uncertain resulthard
- How long will the test takehard
- More notifications, more usershard