Hard Data Scientist questions

32 questions · 20 points each for a correct first answer.

1. 1.Checking results every dayab-testing, experiment-designhard
2. 2.Coaching the worst storescausal-thinking, biashard
3. 3.Unexpected group sizesab-testing, data-qualityhard
4. 4.Using pre-experiment dataab-testing, variance-reductionhard
5. 5.Difference-in-differences assumptioncausal-inferencehard
6. 6.Coefficients that keep flippingregression, model-interpretationhard
7. 7.Feature users retain bettercausal-inference, product-analyticshard
8. 8.Click-through rate standard errorsab-testing, statisticshard
9. 9.Propensity scores near 0 and 1causal-inferencehard
10. 10.Key assumption of an instrumentcausal-inferencehard
11. 11.A link that only appears in a subsetcausal-inference, biashard
12. 12.Error spread grows with the predictionlinear-regression, statisticshard
13. 13.Direction of omitted variable biaslinear-regression, causal-inferencehard
14. 14.AUC after squaring the scoresclassification-metrics, calibrationhard
15. 15.Precision when prevalence dropsclassification-metrics, bayes-theoremhard
16. 16.Oversampling before cross-validationclass-imbalance, data-leakagehard
17. 17.Target encoding without leakagefeature-engineering, data-leakagehard
18. 18.NDCG at 3 for a rankingranking-metricshard
19. 19.Treated drivers take control ridesab-testing, causal-inferencehard
20. 20.Benjamini-Hochberg discoveriesmultiple-testinghard
21. 21.Noisy feature, flatter slopelinear-regression, measurementhard
22. 22.Back-transforming log predictionsregression, forecastinghard
23. 23.Effect with an interaction termlinear-regressionhard
24. 24.From log-odds to probabilitylogistic-regressionhard
25. 25.Shape after NumPy broadcastingnumpyhard
26. 26.Rolling mean at the edgespandas, time-serieshard
27. 27.LEFT JOIN that acts like INNER JOINsql, joinshard
28. 28.Average lifetime with active customerssurvival-analysishard
29. 29.Great backtest, poor live forecasttime-series, data-leakagehard
30. 30.When the bootstrap breaks downbootstrap, statisticshard
31. 31.Optimising time on sitemetrics-design, product-analyticshard
32. 32.Big effect in one segmentab-testing, multiple-testinghard