MLOps Engineer
ML pipelines and reproducibility questions
105 questions on ML pipelines and reproducibility for mlops engineers, easiest first.
Easy 32
- What a model registry is foreasy
- Why version dataeasy
- What experiment tracking recordseasy
- Same code, different resultseasy
- Logging a metric every epocheasy
- Making a training run repeatableeasy
- Bad data reached trainingeasy
- Why pipelines are DAGseasy
- What a Kubeflow component iseasy
- MLflow logging callseasy
- What mlflow.autolog() doeseasy
- What a .dvc file holdseasy
- Training broke after a fresh installeasy
- Bundling preprocessing with the modeleasy
- Feast basic objectseasy
- Two teams, same featureeasy
- Hard-coded settings in scriptseasy
- Testing feature codeeasy
- What continuous training iseasy
- Manual process maturityeasy
- Batch or online scoringeasy
- Splitting a rare-event dataseteasy
- Cron jobs for a training pipelineeasy
- Who approves a production modeleasy
- Documenting a registered modeleasy
- Moving notebook code to productioneasy
- Running notebooks with parameterseasy
- Versioning a model’s interfaceeasy
- Training pod keeps getting evictedeasy
- Database password in config.yamleasy
- What Vertex AI Pipelines runseasy
- How many fits in a grid searcheasy
Medium 43
- When to retrainmedium
- Online and offline feature storesmedium
- Safe retries in pipelinesmedium
- Joining features to training labelsmedium
- Deploying by alias, not versionmedium
- Online features stopped updatingmedium
- Unpausing a DAG with catchup onmedium
- MLflow pyfunc flavormedium
- Why log a model signaturemedium
- Tracing a production modelmedium
- Running dvc repro twicemedium
- Purpose of dvc.lockmedium
- DVC or Git LFSmedium
- Random split for a sales forecastmedium
- A suspiciously strong featuremedium
- Scaling before splittingmedium
- Unseen category at predict timemedium
- Loading the online storemedium
- Features that need request datamedium
- Organising a tuning sweepmedium
- Two runs, two test setsmedium
- Which config was actually usedmedium
- Gating a new modelmedium
- Why export to ONNXmedium
- Artifact was overwrittenmedium
- Reading a table as it wasmedium
- Processing only new datamedium
- Pinning an Iceberg tablemedium
- Which data trained this modelmedium
- Starting before data landsmedium
- Too many tasks hit the databasemedium
- Assets instead of tasksmedium
- Resuming a failed flowmedium
- SageMaker Pipelines step cachemedium
- Training cost on spot GPUsmedium
- Spot instance reclaimedmedium
- SQL offline, Java onlinemedium
- Tumbling or sliding windowmedium
- Label logic changed quietlymedium
- Testing the whole pipeline in CImedium
- Rebuilding the model for productionmedium
- Commit hash with uncommitted editsmedium
- Stopping weak trials earlymedium
Hard 30
- Rollback that made things worsehard
- Same seed, different resultshard
- Loading a model from a public hubhard
- Daily batch missing late eventshard
- Pipeline reused yesterday outputhard
- Streaming counts that run highhard
- Loss spike after resuming traininghard
- Filtering by now() in a taskhard
- Passing a dataframe through XComhard
- Image tag now points elsewherehard
- One customer, both splitshard
- Feature TTL in point-in-time joinshard
- Building a step cache keyhard
- Loading an old picklehard
- Time travel stopped workinghard
- How long a feature backfill takeshard
- Sample changes on every runhard
- Hash split differs between runshard
- SMOTE before cross-validationhard
- Same seed, different data orderhard
- Different GPU count, different resulthard
- Mixing embeddings from two modelshard
- Late events missing from countshard
- Crash after committing offsetshard
- Deletion request vs reproducibilityhard
- Daily full copies of training datahard
- Hashing 50 TB every runhard
- Old run cannot be rebuilthard
- Downstream steps never cachehard
- Bug fixed offline onlyhard