Data Scientist interview questions and answers

I would say: imagine the new feature truly makes no difference. The p-value tells us how often we would see a result at least this big just from random chance in that world. If the p-value is 0.02, results like ours would only happen about 2% of the time by luck, so we have reasonable evidence of a real effect. A common misunderstanding is reading it as "there is a 2% chance the feature does nothing" or "98% chance it works". That is not what it means. Another mistake is treating a small p-value as a big effect. With huge samples, tiny and unimportant differences can be significant. So I always report the effect size with a confidence interval alongside the p-value, and focus on whether the change matters for the business.

I use the median when the data is skewed or has outliers, because the mean gets pulled toward extreme values. For example, delivery time for a food app: most orders arrive in 30 minutes, but a few take two hours because of rain or a lost rider. The mean might say 38 minutes, which does not describe a typical order, while the median of 30 minutes does. Income, house prices and order values are similar. That said, the mean is still important when totals matter, like average revenue per user for forecasting total revenue, since total equals mean times count. Often the best answer is to show the distribution with percentiles, such as the median and the 90th percentile, because the long tail is where customer pain often hides.

Correlation means two things move together. Causation means changing one actually changes the other. Correlation can appear without causation because of a confounder, a third factor that drives both, like hot weather increasing both ice cream sales and drownings. It can also come from reverse causation, where Y actually causes X, or just from chance when we look at many variables. The cleanest way to test causation is a randomised experiment, like an A/B test: randomly give the change to some users, and because the groups are similar in every other way, the difference in outcome is caused by the change. When I cannot run an experiment, I use methods like difference-in-differences or matching, and I am honest that conclusions are weaker and depend on assumptions.

First I find out how much is missing and why. If a value is missing because of a tracking bug on one app version, that is a data quality issue to fix, not just fill. If it is missing for a reason, like "last purchase date" being empty for users who never bought, the missingness itself carries meaning. Then I choose. If very few rows are affected and they look random, dropping them may be fine. For numeric features I often impute the median and add a binary "was missing" flag so the model can learn from it. For categories I add an "unknown" value. Sometimes model-based imputation helps. Many tree models handle missing values directly. For modelling, I fit any imputation on the training set only, to avoid leakage, and check that results do not change much with different choices.

I start with the conclusion, not the method. For example: "Delivery delays are the main reason for the drop in repeat orders, and fixing the evening peak could recover about 4% of revenue." Then I support it with two or three simple charts, each making one point with a clear title. I avoid jargon like p-values and instead say "we are fairly confident the effect is between 2% and 6%". I am honest about limits, such as data gaps or the fact that it is a correlation, not a proven cause. I connect everything to what the audience cares about, like revenue, cost or customer experience, and end with a clear recommendation and next steps. I keep detailed methods in an appendix for anyone who wants to check the work.

Precision asks: of everything the model flagged, how much was really positive? Recall asks: of all real positives, how many did we catch? Most models output a score, and the threshold decides what counts as a positive. If I raise the threshold, I flag fewer cases, so precision usually goes up and recall goes down. Lowering it does the opposite. There is no universally right point. I would plot the precision-recall curve and talk to the business about the cost of each mistake. For fraud review, a missed fraud might cost hundreds of dollars while a false alarm costs a few minutes of an analyst, so I would favour recall up to the team review capacity. For an email that tells customers their account is suspended, false positives are very costly, so I would favour precision. F1 is a handy summary when costs are similar.

The central limit theorem says that if you take many random samples and compute the mean of each, those means form an approximately normal distribution, even when the raw data is not normal, as long as the variance is finite and the sample is reasonably large. Its spread is the standard error, sigma divided by the square root of n. This matters because most of our metrics are averages: average order value, conversion rate, time spent. The CLT is what lets us build confidence intervals and run z or t tests on them without knowing the exact shape of the data. The caveat is that very skewed or heavy-tailed data, like revenue with a few whales, needs a much larger sample before the approximation is good. In those cases I check with a bootstrap, cap extreme values, or use a transformation.

First I find out what one row represents and where the data came from, because the grain decides how everything else is analysed. Then I check basic shape: row counts against what I expect, duplicate rows, and whether the supposed keys are really unique. Next I profile every column: data types, share of missing values, ranges, distinct values and obvious outliers such as negative ages or prices of zero. I check dates cover the period I expect and that there are no gaps or sudden jumps that suggest a pipeline change. I also look at units and currencies. Anything strange I write down and confirm with the data owner rather than guessing. Only after that do I decide how to handle missing values and outliers, and I keep the cleaning steps in code so the analysis is reproducible.

I start with the single message I want the leader to remember, then pick the chart that makes that comparison obvious. A trend over time is a line chart, a ranking across categories is a sorted bar chart, a distribution is a histogram or box plot, and I avoid pie charts unless there are only two or three parts. Then I strip clutter: no 3D, few gridlines, direct labels instead of legends, and one highlight colour for the thing that matters. The title states the takeaway, such as "Churn doubled in the basic plan after the price change", rather than "Churn by plan". I add context like a target line or last year for comparison, and show uncertainty if the numbers are estimates. Finally I check that someone can understand it in ten seconds without me explaining it.

The training set is what the model learns from. The validation set is used to compare models and tune hyperparameters, and because we make many choices with it, its score becomes slightly optimistic. The test set is kept aside and used once at the end to estimate how the model will perform on truly new data. How I split depends on how the model will be used. If it predicts the future, I split by time, training on older data and testing on newer data. If one customer has many rows, I split by customer so the same person never appears in both training and test, otherwise the model can memorise them. For small datasets I use k-fold cross-validation for tuning to get more stable estimates, but still keep a final holdout when possible.

I start with a hypothesis: the new checkout reduces friction, so the purchase conversion rate per visitor will go up. That is my primary metric. I add guardrails like average order value, payment errors, page load time and refund rate, so a "win" does not hide damage. I randomise by user, so people see a consistent page across sessions. Then I run a power analysis using the baseline conversion, its variance and the smallest lift worth detecting to decide sample size, and I run for at least one or two full weeks to cover weekday patterns. During the test I check for sample ratio mismatch and logging problems, but I do not stop early just because the p-value looks good. Before launch, the team agrees on the decision rule, for example ship if conversion improves significantly and no guardrail worsens.

I start with the goal: help users find products they want, which should increase purchases. Clicks on the section are easy to measure but can be a vanity metric, since eye-catching items get clicks without sales. So I would choose something closer to value, like purchases or revenue per user that come from the section, or better, overall purchases per user, because the section could just steal purchases from search. That overall metric captures cannibalisation. I would add guardrails: return rate, page load time, and diversity of items shown. The metric must also be sensitive enough to move within a few weeks of testing; if overall revenue is too noisy, I might use a proxy like add-to-cart rate, after checking that it predicts revenue. Longer term, I would watch repeat visits.

First I clean up: remove features that leak the target, such as anything recorded after the churn date, plus constant, nearly empty and duplicate columns. I check pairs with very high correlation and keep the more reliable one. Domain knowledge helps me group features into themes, like usage, billing and support. Then I let a model help. With gradient boosting I look at permutation importance or SHAP values on validation data, since built-in split importance can be misleading. L1 regularisation in a logistic model is another way to shrink weak features to zero. I always judge choices by cross-validated performance, not training scores, and check the selection is stable across folds. Finally I consider practicality: features that are expensive or slow to compute in production, or hard to explain, need to earn their place.

I would explain at two levels. Globally, I show which features matter most and how they affect risk, using SHAP summary plots or partial dependence plots, for example "risk rises steadily with the number of missed payments". I check with credit experts that these patterns make sense. Locally, for individual applicants, SHAP values show which factors pushed a score up or down, which also helps with adverse action reasons customers may be owed. Trust also comes from evidence: I show performance on recent data, calibration, so a predicted 10% default rate really means about 10%, and results across segments to check fairness. I walk through real examples, including mistakes. If the gain from a complex model is small, I would consider a simpler scorecard or logistic model that is easier to explain and govern.

First I define churn clearly, for example "no purchase in 60 days" or "subscription cancelled", and check how the rate has changed over time using retention curves by signup cohort. Then I segment: by plan, acquisition channel, region, device and tenure, to see where churn is concentrated. Often one segment, like users from a discount campaign, explains most of it. Next I look at behaviour before churn, such as fewer logins, support tickets or failed payments, and combine this with qualitative data like cancellation reasons and support conversations. A churn model with SHAP can show which factors predict churn, but I am careful to say these are associations, not proven causes. I finish with prioritised recommendations, such as fixing failed payment retries, and propose experiments to confirm the biggest drivers.

First I would make sure evaluation is honest. Accuracy is useless at 0.5% positives, so I would use PR-AUC and a business metric such as recall at the precision the operations team can handle. I would use stratified splits, or time-based ones if relevant, and keep the true class balance in validation and test. For training, I would start with a strong baseline like gradient boosting with class weights, which often works as well as resampling. If I try undersampling or SMOTE, I apply it only inside training folds to avoid leakage. Then I choose the threshold from the precision-recall curve based on the cost of a missed positive versus a false alarm. If the scores feed pricing or prioritisation, I check calibration, because weighting and resampling distort probabilities, and recalibrate with isotonic or Platt scaling if needed.

I would run a rolling-origin backtest: train on data up to a cutoff, forecast the next horizon, move the cutoff forward and repeat, so every test period is in the future relative to training. I would always compare against simple baselines like last year same week or a seasonal naive forecast, because a complex model that cannot beat them is not worth running. For metrics I would use WAPE or MAE for accuracy and check bias separately, since a forecast that is always 5% low causes stock-outs even if its error looks small. I would check every feature is available at forecast time, for example using promotions that are planned in advance but not actual sales. Finally I would break errors down by product, store, horizon and holiday periods, and share those results with planners so they know where to trust the forecast.

Leakage is when the model learns from information that will not be available when it makes real predictions, so offline scores look great but production results do not. Warning signs are scores that seem too good, like 0.99 AUC on a hard problem, or one feature dominating importance. I investigate those features and ask when each value becomes known. Common causes are features derived from the label, such as a refund flag in a fraud model, using future data in time series, fitting scalers or encoders on the whole dataset before splitting, and the same customer appearing in both train and test. To prevent it I build features with point-in-time logic using an as-of timestamp, put all preprocessing inside a pipeline that is fitted on training folds only, split by time or entity when needed, and finally compare offline metrics with a short shadow run in production.

I usually try both, because the comparison itself is informative. A regularised linear or logistic model is fast, stable, easy to explain with coefficients and works well when data is small, very wide or sparse, such as text features. It needs manual work for non-linear effects and interactions. Gradient boosted trees usually win on accuracy for medium to large tabular data because they find non-linearities and interactions automatically and handle missing values and mixed scales. The trade-offs are less transparent explanations, more tuning and a higher risk of overfitting on small data. I would also weigh the context: if a regulator needs clear reasons, or the gain from boosting is tiny, I prefer the linear model. I compare them on the same validation scheme against a simple baseline, and look at calibration and stability across time periods, not only the headline metric.

I start with the primary metric and get its baseline value and variance from historical data. Then I agree a minimum detectable effect with the product team, meaning the smallest change worth acting on, not the effect we hope for. With alpha usually 0.05 and power 80%, I compute the sample size per group; for a conversion rate this is roughly 16 times p(1-p) divided by the effect squared. I divide that by the daily eligible traffic to get the number of days, then round up to full weeks so weekday and weekend behaviour are both included. If the duration is too long, I look at variance reduction such as CUPED, a more sensitive metric, or a larger effect size. Finally I write the plan down before launch, including the stopping date, so we do not peek and stop early on a lucky result.

First I check the test is valid: no sample ratio mismatch, no logging changes, enough duration to pass novelty effects, and the metrics were planned in advance. Then I look at what actually matters for the business, usually revenue or profit per visitor, which combines conversion and order value. A 3% rise in conversion and a 2% fall in order value is roughly a 1% revenue gain, but I would compute it directly with a confidence interval. Next I segment to understand the mechanism: maybe the new design brings in more small first-time purchases, which could be good for long-term retention, or maybe it pushes cheaper items with lower margin. I would check margin, return rate and repeat purchase rate. My recommendation would explain the trade-off clearly and might suggest a follow-up test to keep the conversion gain while protecting order value.

I start by defining the question exactly: the effect of joining, or of the programme existing, on spend over what time frame. Then I look for natural comparisons. If the programme rolled out in different cities at different times, difference-in-differences comparing early and late cities works, after checking parallel pre-trends. If there was an eligibility rule, like spending above a threshold, regression discontinuity compares customers just above and below. If all customers got it at once, synthetic control or interrupted time series with a forecast of the counterfactual can help, using similar markets or products as controls. Comparing members to non-members needs matching on pre-launch behaviour, but self-selection bias remains. I run placebo tests, such as fake launch dates, and try several methods. I report a range, not one number, and state the assumptions clearly.

First I learn how the forecast is used: if it drives store replenishment, I need daily forecasts per product and store, a horizon matching lead times, and the cost of overstock versus stockouts. I build simple baselines first, like seasonal naive using the same weekday last week. Then I train one global model, often gradient boosting, across all series with features such as lagged sales, rolling averages, day of week, holidays and festivals, price, promotions and store attributes. A global model shares patterns, which helps sparse products. I treat stockouts carefully, since zero sales during a stockout are not zero demand. For new products I use similar-item features. I evaluate with rolling time-based backtests using metrics like weighted MAPE, and produce quantile forecasts because replenishment needs uncertainty, not only a point estimate.

In a marketplace, riders in treatment and control compete for the same drivers. If a new pricing feature makes treatment riders book more, they take drivers away from control riders, so control gets worse and the measured lift is exaggerated. This breaks the assumption that one user outcome does not depend on another group. To reduce interference, I randomise at a level where spillover is small. Switchback experiments alternate treatment and control for a whole city or zone over time windows, like 30 minutes, so the whole market is in one state at a time. Cluster randomisation by city or region also works. The trade-off is fewer independent units, so less power and wider confidence intervals; I need longer tests and careful variance estimation. I would also measure both sides: rider conversion and wait times, and driver earnings and utilisation.

First I check whether the drop is real. I look for pipeline delays, missing partitions, changes in event tracking or a new metric definition, and compare with other sources like server logs or backend orders. If the data is fine, I check the calendar and outside events: a festival, a public holiday, a payment or app store outage. Then I segment: platform, app version, country, acquisition channel, new versus returning users. A sharp drop limited to one segment usually points to the cause, like Android users on the latest release, which suggests a bug in that version, or one region, suggesting an outage or network issue. I check release notes, feature flags and experiments launched that day. I share a short update early with what we know and do not know, then the root cause and fix once confirmed.

The trap is to target people with a high purchase probability, because many of them would buy anyway and the discount just loses margin. What we want is uplift: how much the discount raises the chance of buying for each customer. To learn that we need randomised data, so I would run an experiment where a random subset gets the discount. Then I can fit uplift models, for example a T-learner with one model for treated and one for control, or a causal forest, and estimate each customer effect. Evaluation is tricky because we never see both outcomes for one person, so I use uplift or Qini curves on held-out randomised data, checking that the top-scored groups show the largest treated-minus-control difference. Finally I would run a confirmation test comparing model targeting with random targeting, measured on incremental profit after discount costs.

First I confirm the drop is real: are labels complete yet, is the metric computed the same way as offline, and did the population change, for example a new market? Then I look for training-serving skew, which is the most common cause. I log the features the model actually received and compare them with what the offline pipeline would compute for the same entities; differences in units, time zones, default values or missing joins show up quickly. Next I compare feature and score distributions with training data using PSI or simple histograms to find data drift. If inputs look the same but performance fell, the relationship may have changed, which is concept drift, perhaps from a competitor or policy change. I also revisit the offline evaluation for leakage, since a too-good backtest often explains a disappointing launch, and check for feedback loops where the model changes the data it later learns from.

A binary classifier answers "will this customer churn in the next 30 days", which needs a fixed window and throws away information about when churn happens. Survival analysis models time to churn and uses customers who are still active as right-censored observations, rather than dropping them or treating them as non-churners. I would start with Kaplan-Meier curves by plan or acquisition channel to describe retention. For prediction I would use a Cox proportional hazards model for interpretable hazard ratios, or a survival forest or gradient boosted survival model if effects are non-linear. I would check the proportional hazards assumption, because effects like a discount often fade over time, and handle time-varying covariates such as recent usage carefully to avoid using future data. Evaluation uses the concordance index for ranking and calibration of predicted survival at key horizons like 90 days.

Propensity methods only adjust for confounders we measure, so the biggest risk is unmeasured ones like motivation, which drives both adoption and retention. I would list likely confounders with the product team and include pre-treatment behaviour such as usage in the weeks before adoption, which is a strong proxy for motivation. I must avoid post-treatment variables, because adjusting for things the feature itself changes removes part of the real effect or adds bias. Next I check overlap: if many adopters have propensities near one, there are no comparable non-adopters, so I trim and state the population the estimate applies to. After matching or weighting I check balance with standardised mean differences below about 0.1. Finally I run a sensitivity analysis to see how strong a hidden confounder would need to be to remove the effect, and use a negative control outcome the feature should not affect.

The core problem is confounding: prices are often cut when demand is low or raised during peaks, so a naive regression of sales on price gives the wrong sign or size. The best data comes from randomised price tests across stores or time, and the next best from natural variation like staggered price changes or cost-driven changes unrelated to demand. I would model log units against log price, so the coefficient is the elasticity, with controls for seasonality, promotions, competitor prices and holidays. Stock-outs must be handled, because zero sales when out of stock is not low demand. With thousands of products, many have little price variation, so I would use a hierarchical model that shrinks each product towards its category average. I would also check cross-effects, since cutting one price can cannibalise similar products, and validate predictions on held-out price changes before using them for pricing.