medium10 pts · first answer countssparkspark-iopartition-pruning

Filtering on a partition column

A table is written with partitionBy("dt"), one folder per day for 3 years. A query filters WHERE dt = '2026-03-01'. What does Spark read?

A.All files, then drops other dates in memoryB.Only the first file of each folderC.All files, but only the dt column from eachD.Only the files in the dt=2026-03-01 folder

Discussion 0

Join free to post in the discussion.

Filtering on a partition column

Discussion 0

Related questions