Skip to content
Skills
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Problems
Questions
Roles
Skills
Interview
Contest
Leaderboard
More
Home
/
Data Engineer
/
Questions
/
Spark Tuning
Data Engineer
Spark Tuning questions
15 questions on Spark Tuning for data engineers, easiest first.
Medium
7
Why a groupBy made 200 tasks
medium
How many waves of tasks
medium
An iterative loop that gets slower
medium
Sorting rows inside each file
medium
Removing idle executors safely
medium
Faster serialization for RDDs
medium
Concurrent tasks with task.cpus 2
medium
Hard
8
Executors on a 16-core node
hard
Unified memory for a 10 GB heap
hard
Container killed for exceeding memory
hard
Adding 300 columns with withColumn
hard
Speculation and a database sink
hard
Choosing partitions for a 500 GB shuffle
hard
Bad join order on Hive tables
hard
coalesce(1) made the whole job slow
hard
