Skip to content
Skills
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Home
/
MLOps Engineer
/
Questions
/
Parallelism
MLOps Engineer
Parallelism questions
8 questions on Parallelism for mlops engineers, easiest first.
Medium
1
How tensor parallelism splits a layer
medium
Hard
7
Idle time in pipeline parallelism
hard
Why 1F1B beats GPipe on memory
hard
Training on 128K-token sequences
hard
Placing TP, PP and DP on a cluster
hard
Tensor parallel size beyond 8
hard
Splitting a model across two nodes
hard
Placing experts on many GPUs
hard
