easy5 pts · first answer countssparkspark-partitions
Tasks in a 64-partition stage
A stage reads a DataFrame that has 64 partitions and does only narrow work such as filter and select. How many tasks does that stage run?
Discussion 0
Join free to post in the discussion.
Related questions
- When Spark actually runsData Engineer
- What a Spark partition isData Engineer
- What the Spark driver doesData Engineer
- Calling collect() on 500 GBData Engineer
- Why DataFrames beat RDDs of objectsData Engineer
- Entry point of a Spark 3+ appData Engineer
- Where a job splits into stagesData Engineer
- Cost of inferSchema on CSVData Engineer