medium10 pts · first answer countssparkspark-iodata-layout

partitionBy on user_id

A developer writes a table with partitionBy("user_id") for 5 million users. What problem follows?

A.Spark refuses to partition by a string columnB.All data goes to one folder because of hashingC.Millions of folders of tiny files that are slow to readD.Queries that filter by date become much faster

Discussion 0

Join free to post in the discussion.

partitionBy on user_id

Discussion 0

Related questions