hard20 pts · first answer countswindow-functionssqlspark

Spark window with no partition

In Spark, a ROW_NUMBER over Window.orderBy("event_ts") with no partitionBy runs on a 2-billion-row table. What happens?

A.All rows are moved to a single partition and one task does all the workB.Spark splits the window across all executors automaticallyC.Spark refuses to run and throws an error at compile timeD.Each executor numbers its own rows starting at 1

Discussion 0

Join free to post in the discussion.

Spark window with no partition

Discussion 0

Related questions