medium10 pts · first answer countssparkspark-sql

Making pivot faster

df.groupBy("store").pivot("month").sum("sales") runs an extra job before the main one. How do you avoid it?

A.Cache the DataFrame before pivotingB.Use orderBy("month") before the pivotC.Replace sum with avg in the aggregationD.Pass the list of months: pivot("month", months)

Discussion 0

Join free to post in the discussion.

Making pivot faster

Discussion 0

Related questions