hard20 pts · first answer countsexperimentationguardrail-metrics

A winning test with a hidden cost

A new checkout design increased conversion by 5%, but the refund rate rose by 15% in the test group. What should you do?

A.Ship it; conversion is the primary metricB.Ignore refunds because they happen laterC.Stop A/B testing because results are confusingD.Do not ship yet; investigate why refunds rose, calculate net revenue and customer impact, and adjust the design

Discussion 0

Join free to post in the discussion.

A winning test with a hidden cost

Discussion 0

Related questions