easy5 pts · first answer countsspark
When Spark actually runs
In Spark, which of these lines actually starts a job on the cluster?
df = spark.read.parquet("s3://bucket/sales") # 1
df2 = df.filter(df.amount > 100) # 2
df3 = df2.select("region", "amount") # 3
n = df3.count() # 4Discussion 0
Join free to post in the discussion.
Related questions
- What a Spark partition isData Engineer
- What causes a shuffleData Engineer
- Joining with a small lookup tableData Engineer
- When to cache in SparkData Engineer
- repartition vs coalesceData Engineer
- One task takes foreverData Engineer
- Which late event is keptData Engineer
- A slow Python UDF in PySparkData Engineer