Machine Learning Engineer interview questions and answers

Overfitting is when a model learns the training data too closely, including its noise, so it performs well on training data but poorly on new data. I detect it by comparing training and validation metrics: if training error keeps going down while validation error goes up, the model is overfitting. Learning curves make this easy to see. To reduce it, I first try to get more or cleaner data. Then I add regularisation such as L1 or L2 penalties, dropout for neural networks, or limits like max depth for trees. Early stopping based on validation loss is simple and effective. I may also reduce model complexity or features. I use cross-validation to pick these settings so the choice is not tuned to one lucky split, and I keep a separate test set for the final check.

Precision is the share of predicted positives that are truly positive. Recall is the share of actual positives that the model finds. Take a cancer screening model: recall matters most, because missing a sick patient (a false negative) is very costly, while a false positive only leads to an extra test. For a spam filter, precision matters more, because sending an important email to spam is worse than letting a few spam emails through. Most models output a score, and moving the decision threshold trades precision against recall: a lower threshold catches more positives but adds false alarms. I pick the threshold using validation data and the real business cost of each error type. When I need one balanced number, I use F1 score, and to compare models across thresholds I use the precision-recall curve.

Gradient descent is a way to minimise a loss function. We compute the gradient of the loss with respect to each parameter, which points in the direction of steepest increase, and then move the parameters a small step in the opposite direction. We repeat this until the loss stops improving. The learning rate controls the size of each step. If it is too high, updates overshoot and the loss can oscillate or explode. If it is too low, training is very slow and can get stuck. In practice we use mini-batch stochastic gradient descent, computing gradients on small batches, which is faster and adds helpful noise. Optimisers like Adam adapt the step size per parameter, and schedules such as warmup followed by cosine decay often make training more stable.

The training set is what the model learns its parameters from. The validation set is used to compare models and tune hyperparameters such as learning rate or tree depth. The test set is held back until the very end and used once to estimate how the final model will perform on new data. If I keep checking the test set while tuning, it stops being unbiased. How I split depends on the data. For classification with imbalanced classes I use a stratified split so each set has similar class ratios. For time-based data I split by time, training on the past and testing on the future. If the same user or patient appears many times, I split by group so the same person is not in both train and test, which would leak information.

A baseline tells me what "good" looks like. Without it I cannot say whether a fancy model is actually helping. For classification, the simplest baseline is predicting the majority class; for regression it is predicting the mean or the last value. A step up is a simple model such as logistic regression or a small gradient boosted tree, or a business rule the team already uses. Baselines also help catch bugs: if my deep model barely beats predicting the mean, I probably have a data or pipeline problem. Finally, complex models cost more to train, serve and maintain, and are harder to explain. If a simple model gets 95% of the value, it is often the right production choice. I always report new results against the baseline on the same validation data.

Bias is the error that comes from a model being too simple to capture the real pattern, like fitting a straight line to a curve. It shows up as poor scores on both training and validation data, which is underfitting. Variance is how much the model changes when trained on a different sample. High variance means it memorises noise, so training scores are great but validation scores are poor, which is overfitting. As model complexity rises, bias usually falls and variance rises, so total error is lowest somewhere in the middle. I find that point with learning curves and cross-validation. To reduce variance I add data, regularisation such as L2 or dropout, or use bagging. To reduce bias I add features, use a more flexible model or train longer.

An embedding is a list of numbers, a dense vector, that a model produces for a piece of text, an image, a user or a product. The model is trained so that things with similar meaning or behaviour end up close together in that space. I usually compare embeddings with cosine similarity. In a product, a common use is semantic search: embed every help article once, store the vectors in a vector index, then embed the user query and return the nearest articles, even when they share no exact words. The same idea powers related-item recommendations, duplicate detection, clustering of support tickets and retrieval for RAG. Before shipping I test the embedding model on real queries from our domain, because a general model can miss jargon, and I often add keyword search for exact codes.

For tabular data with mixed features, gradient-boosted trees are often my first strong model. They find non-linear relationships and feature interactions on their own, handle missing values, and do not need feature scaling. They usually win on accuracy for this kind of data. I would pick a linear or logistic regression when the relationship is close to linear, when I need very clear explanations, for example in regulated credit decisions, when data is small, or when latency and model size must be tiny. Linear models also extrapolate, while trees predict flat values beyond the range they saw in training, which matters for trends over time. In practice I start with a simple linear baseline, then try boosting, and choose using cross-validated metrics together with interpretability and serving cost.

I start by asking why the values are missing. If a field is missing because a user skipped an optional step, the fact that it is missing may itself predict the target. If it is a logging bug, I fix the pipeline instead of patching the model. For numeric features I often impute the median, which is robust to outliers, and for categories I use the most frequent value or a separate "unknown" category. I usually add a binary indicator column so the model can learn from the missingness. All imputers are fitted only on the training split inside a pipeline, so validation and serving use the same learned values without leakage. Gradient boosting libraries can handle missing values natively, which is often the simplest option. If a column is mostly empty and adds little, I drop it.

Self-attention lets every token look at every other token and decide which ones matter for it. Each token embedding is projected into three vectors: a query, a key and a value. To update one token, we take the dot product of its query with every key, divide by the square root of the key size to keep values stable, and apply softmax to get attention weights that sum to one. The new representation is the weighted sum of the value vectors. So in "the animal did not cross the street because it was tired", the token "it" can put high weight on "animal". Multi-head attention runs several of these in parallel so different heads can capture different relations. In decoder models a causal mask stops tokens from attending to future positions.

First I would agree on the right metric. Accuracy is useless here, so I would use PR-AUC and recall at a fixed precision, or a cost-based metric using the real cost of missed fraud versus blocked good customers. I would split by time, because fraud patterns change and the model will predict the future. I would also check label delay, since chargebacks can arrive weeks later. For training, gradient boosting works well on tabular fraud data. I would handle imbalance with class weights or by undersampling negatives in the training set only, and then correct the probabilities if needed. After training, I would choose a threshold on validation data that matches the review team capacity or cost trade-off. Finally I would do error analysis on false positives and false negatives to find new features, and monitor drift after launch.

A feature store gives one place to define, compute and serve features. The main benefit is consistency: the same feature logic feeds both training and online serving, which prevents training-serving skew. It also produces point-in-time correct training sets, so each training row only uses feature values that were known at that moment, avoiding leakage. Usually it has an offline store, such as a warehouse table, for building training data, and an online store, such as a key-value database, for low-latency lookups at prediction time. It also helps teams discover and reuse features instead of rebuilding them. It is worth adding when several models share features, when you serve real-time predictions, or when skew bugs have hurt you. For one batch model owned by a small team, a well-tested SQL pipeline may be enough.

I start with the product need. If predictions can be a few hours old and the set of entities is known, like daily churn scores for all customers, batch inference is simpler and cheaper: run a scheduled job, write scores to a table and let other systems read them. If the prediction depends on what is happening right now, like fraud checks during payment or search ranking for a new query, I need online inference behind an API with tight latency limits. Online serving adds work: autoscaling, low-latency feature lookups, monitoring and fallbacks. Many systems are hybrid: batch-compute heavy features or candidate lists, then do a light online model at request time. I also consider cost per prediction, peak traffic and what happens if the model service is down, and design a safe fallback.

I start with the data. I look at a few batches to confirm inputs and labels match, are normalised and are not all the same, and that augmentation has not broken them. Next I try to overfit a tiny batch of 10 to 50 examples. If the model cannot drive loss close to zero on that, there is a bug in the model, loss or training loop. I check common mistakes: wrong loss for the task, applying softmax twice, forgetting optimizer.zero_grad(), or a frozen layer. Then I sweep the learning rate, since too high or too low are both common causes. I log gradient norms per layer to spot vanishing or exploding gradients, and use gradient clipping or better initialisation if needed. Finally I simplify: smaller model, no regularisation, and compare against a simple baseline to rebuild step by step.

I monitor at three levels. First, system health: latency, error rate, throughput and resource use, like any service. Second, data: distributions of each input feature, missing values, and new categories, compared against the training data using measures like population stability index or KL divergence. I also track the distribution of predictions, since a sudden shift often means an upstream bug. Third, model quality: once labels arrive, I compute the real business metric and model metrics over time, sliced by important segments. When drift appears, I first check whether it is a pipeline bug, such as a changed unit or a broken join. If the world has truly changed, I retrain on recent data, validate against the current model, and roll out gradually with a canary or shadow deployment. Scheduled retraining plus alerts covers most cases.

I start with ingestion: parse documents, strip boilerplate, and split them into chunks of a few hundred tokens along headings, keeping metadata such as source, date and permissions. I embed each chunk and store it in a vector index, and I add BM25 keyword search because users often search exact product names or codes. At query time I retrieve candidates from both, merge them, and run a cross-encoder reranker to pick the top few that fit the context budget. The prompt tells the model to answer only from the provided passages, cite them, and say when it does not know. Before launch I build an evaluation set of real questions with known source passages, measure retrieval recall at k and answer faithfulness, and iterate on chunking and retrieval first. In production I enforce document permissions at retrieval time, re-index on updates, and monitor no-answer rates and user feedback.

First I define what good looks like for this feature, for example a correct answer, the right JSON format, no made-up facts and a polite tone. I build a test set from real or realistic inputs, including hard and adversarial cases, with reference answers where possible. Some checks are automatic: schema validation, exact match on extracted fields, and whether answers are supported by the retrieved context. For open-ended quality I use an LLM judge with a clear rubric, and I check its agreement with human ratings on a sample and control for position and length bias. This suite runs on every prompt or model change so I catch regressions. After launch I run an A/B test or staged rollout and track task success, thumbs-up rates, escalations, latency, cost per request and safety incidents, and I feed failures back into the test set.

I start from sensible defaults from the library or a similar published setup and get a baseline score. Then I focus on the parameters that usually matter most. For neural networks that is the learning rate, then batch size, weight decay and schedule. For gradient boosting it is learning rate, number of trees with early stopping, depth and regularisation. I search on a log scale for things like learning rate and use random search or Bayesian optimisation rather than a grid. To save compute I use early stopping inside each trial and methods like successive halving or Hyperband that stop weak trials early. I often tune on a subset of data first, then confirm the best settings on the full data. All decisions use validation or cross-validation, and the test set is used once at the end.

I start with the cheapest option: a good model with a clear prompt and a few examples, measured on an evaluation set of real questions. If the failures are about missing or out-of-date knowledge, which is likely for internal products, I add RAG. It keeps facts in documents that can be updated daily, supports citations and respects permissions. Fine-tuning is better when the problem is behaviour: a strict output format, a domain style, or a narrow task where a smaller fine-tuned model could replace a big one to cut latency and cost. I would use parameter-efficient methods like LoRA and need a few thousand quality examples. Fine-tuning is a poor way to inject facts that change often. Often the answer is RAG plus a little fine-tuning, chosen by comparing options on the same evaluation set.

A sudden big jump makes me suspicious before it makes me happy. I first look at feature importance or SHAP values and examine the top features: is any of them created after the outcome, such as a "refund issued" flag in a churn model, or a proxy for the label? Next I check the split. For time-based problems, training data must come strictly before validation data, and rows from the same user or entity must not appear on both sides. I look for duplicate or near-duplicate rows across splits. Then I check the pipeline: scalers, imputers, target encoders and resampling must be fitted only on training folds. Finally I score the model on a clean holdout from a later period, or run it in shadow mode on live traffic. If the gain disappears there, it was leakage.

I would use a multi-stage system. First, candidate generation picks a few hundred products from 10 million within tens of milliseconds. A two-tower model learns user and product embeddings, and I retrieve neighbours with an approximate nearest neighbour index, alongside simple sources like trending items and recently viewed categories. Second, a ranking model, such as gradient boosting or a deep model, scores candidates using user history, product features, price, context and cross features, trained on clicks and purchases with weights favouring conversions. Third, re-ranking adds diversity, removes out-of-stock items and applies business rules. For cold start I use content features and popularity for new products and new users. Features come from a feature store to avoid skew. I evaluate offline with recall and NDCG, but trust online A/B tests on conversion and revenue, keeping the total latency under around 150 milliseconds.

First I make sure I can see what production sees: log the exact feature vectors and predictions for a sample of requests. Then I recompute the same rows with the offline pipeline and compare feature by feature. Large differences point to the cause. Typical causes are separate code for training and serving, such as SQL versus a Java service, different handling of missing values, unit or timezone differences, and stale caches. Another cause is leakage offline: training features built with data from after the prediction time, which makes offline scores too good. To prevent it, I use one feature definition for both paths, often through a feature store, and build training data from logged features or point-in-time joins. I add unit tests that compare both paths on fixed examples, and monitoring that alerts when serving feature distributions move away from training.

I first profile to find the real bottleneck: compute, memory or input pipeline. For memory, mixed precision with bf16 and activation checkpointing are cheap wins. If the model still does not fit, I use fully sharded data parallel or ZeRO, which shards parameters, gradients and optimizer states across GPUs. For very large models I add tensor parallelism within a node, where interconnect is fast, and pipeline parallelism across nodes. For speed, data parallelism across many GPUs scales throughput, but a larger global batch may need learning rate warmup and tuning to keep quality. Communication becomes the limit at scale, so I overlap it with compute and use fast interconnects. Long runs fail, so I save sharded checkpoints regularly, make jobs resumable, and track loss and gradient norms to catch instability early.

Before the test I run the new model in shadow mode to check latency, errors and prediction distributions without affecting users, then a small canary. For the A/B test, I define one primary metric, like conversion rate, and guardrail metrics such as latency, revenue per user and complaint rate. I do a power analysis to pick the sample size and run for at least one or two full weekly cycles. I randomise by user, not by request, so each person has a consistent experience. Pitfalls include peeking and stopping early when results look good, which inflates false positives; novelty effects that fade; interference when users affect each other, like marketplaces; and feedback loops where the model changes the data it later trains on. I also check sample ratio mismatch and segment results, and only ship when the gain is significant and guardrails hold.

First I profile the full request path, because often the slow part is feature lookups or preprocessing, not the model. For the model itself, I try quantisation to int8 or lower precision, which can give large speedups with small accuracy loss. Knowledge distillation trains a smaller student model to copy the large one, often keeping most of the quality. Pruning can help on supported hardware. I export to an optimised runtime like ONNX Runtime or TensorRT, which fuses operations. On the serving side, dynamic batching improves GPU use, caching handles repeated inputs, and choosing the right instance type or moving small models to CPU can cut cost. Sometimes a cheaper model handles most requests and only hard cases go to the large one. For every change I measure accuracy on the validation set and latency at p95 and p99, not just average.

I would start with a modern serving engine that supports continuous batching, so new requests join the running batch at each decoding step instead of waiting for a whole batch to finish. Memory is the main limit, so I use a paged KV cache to avoid fragmentation and prefix caching for shared system prompts. Quantising weights to 8 or 4 bits, and the KV cache where quality allows, frees memory for more concurrent sequences. Fused attention kernels cut memory traffic. For latency I track time to first token and time per output token separately, cap the maximum batch size to protect tail latency, and consider speculative decoding. I might route easy requests to a smaller model. Finally I load test with realistic prompt and output lengths, set autoscaling on queue depth and KV cache usage, and monitor p50, p99 and cost per thousand tokens.

I assume the model can be manipulated, so safety cannot rely on the prompt alone. Each tool gets the narrowest permissions possible, and calls run with the end user's credentials, so the agent can never see more data than that user can. Retrieved documents, emails and web pages are treated as untrusted data that may contain injected instructions. I validate tool arguments against schemas and allow-lists, and actions with real impact, such as sending money, emailing outside the company or deleting records, need explicit human confirmation. Input and output filters check for harmful content, secrets and personal data before anything leaves the system. Everything is logged for audit. Before launch we red team the agent with injection and jailbreak attempts, turn those into a regression suite, and keep running it on every model or prompt change.

First I measure instead of guessing. I profile a few steps on 8 and 64 GPUs and break step time into data loading, forward and backward compute, and communication. If GPU utilisation is low and there are gaps before each step, the data loader is the bottleneck, and I add workers, prefetching or faster storage. If communication dominates, I look at gradient all-reduce or FSDP all-gather time. Across nodes the network is much slower than within a node, so I check interconnect bandwidth, make sure communication overlaps with backward compute, use bucketed gradients, and increase work per step through larger per-GPU batches or gradient accumulation. I also look for stragglers: one slow node or uneven sequence lengths make everyone wait. After each change I re-measure scaling efficiency and keep the learning rate tuned for the new global batch.

Fraud is rare, often well under 1% of transactions, and both missed fraud and blocked good customers are costly, so I optimise expected cost and use precision-recall metrics. The decision must happen in tens of milliseconds. Strong signals are behavioural, such as how many transactions a card made in the last hour, new device, or distance from usual locations, so I compute these in a streaming feature store that serves the same definitions online and offline. A gradient-boosted model scores each transaction, combined with hard rules for known patterns, and thresholds map scores to approve, challenge or block based on cost. Labels arrive late through chargebacks, and fraudsters adapt, so I validate on later time periods, retrain often and monitor score distributions and fraud rates. New models run in shadow mode before taking over.

I start by setting a quality bar on our own evaluation set and measuring the full-precision model as a baseline. Then I try the least risky option first, usually 8-bit weight-only quantisation, which often has almost no quality loss and halves memory. If I need more savings I try 4-bit weights with group-wise scales using a post-training method that uses calibration data. That data should look like real production inputs. Quantising activations as well gives more speed, but outlier channels can hurt accuracy, so I use per-token or per-channel scales or keep sensitive layers in higher precision. I evaluate on task metrics and edge cases, not just perplexity, since small averages can hide failures. Finally I benchmark latency, throughput and memory on the actual serving hardware, because the speed-up depends on kernel support.