medium10 pts · first answer countsexperimentationstatistics
Detecting a smaller effect
A test needs about 20,000 users per variant to detect a 10% relative lift. You now want to detect a 5% lift with the same confidence and power. Roughly how many users per variant do you need?