hard20 pts · first answer countsdata-pipelinesbackfillsconcurrency
Backfill and daily load collide
A backfill rewrites January–March while the normal daily job keeps writing today. Both write the same summary table with a delete-then-insert per month. March totals come out wrong. What is the safest approach?
Discussion 0
Join free to post in the discussion.
Related questions
- ETL vs ELTData Engineer
- Idempotent pipelinesData Engineer
- Batch or streaming for a daily reportData Engineer
- Incremental logic for a tiny tableData Engineer
- What SCD Type 1 doesData Engineer
- Keeping the old customer segmentData Engineer
- Why a dimension has its own keyData Engineer
- The bronze layerData Engineer