hard20 pts · first answer countsab-testingcontent-analytics

Calling a headline test too early

Headline A got 15 clicks from 500 views (3.0%). Headline B got 18 clicks from 500 views (3.6%). The team wants to declare B the winner. What should you say?

A.B wins by 20%, so ship itB.A wins because it was shown firstC.Average the two and keep bothD.A difference of 3 clicks is well within normal random variation; keep testing to a planned sample size before deciding

← Seasonal dip or content decay?Playful voice in a payment error →