Skip to content
Skills
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Home
/
MLOps Engineer
/
Questions
/
Model evaluation
MLOps Engineer
Model evaluation questions
17 questions on Model evaluation for mlops engineers, easiest first.
Easy
3
Data used to tune hyperparameters
easy
Recall from prediction counts
easy
Splitting a rare-event dataset
easy
Medium
8
Joining features to training labels
medium
Automatic gate before promotion
medium
Random split for a sales forecast
medium
A suspiciously strong feature
medium
Scaling before splitting
medium
Reading a reliability diagram
medium
Brier score by hand
medium
Precision moved, AUC did not
medium
Hard
6
One customer, both splits
hard
Hash split differs between runs
hard
SMOTE before cross-validation
hard
Computing ECE
hard
AUC steady, losses rising
hard
Updating a score for a new base rate
hard
