hard20 pts · first answer countsairflowidempotency

Two runs overwriting one temp file

With max_active_runs=4 during a backfill, a task writes /tmp/extract.csv on a shared volume and the next task loads it. Some days load another day's data. What is the fix?

A.Set retries=3 on the load taskB.Use a pool with 4 slotsC.Raise execution_timeout on both tasksD.Include the run id or ds in the file path

Discussion 0

Join free to post in the discussion.

Two runs overwriting one temp file

Discussion 0

Related questions