hard20 pts · first answer countsdata-pipelinesincremental-loads
A row missed despite >= on updated_at
A loader runs every 10 minutes with updated_at >= last_max. A source transaction set updated_at = 10:02 but committed at 10:13. The 10:10 run saw max 10:09. Why was the row never loaded?
Discussion 0
Join free to post in the discussion.
Related questions
- ETL vs ELTData Engineer
- Idempotent pipelinesData Engineer
- Batch or streaming for a daily reportData Engineer
- Incremental logic for a tiny tableData Engineer
- What SCD Type 1 doesData Engineer
- Keeping the old customer segmentData Engineer
- Why a dimension has its own keyData Engineer
- The bronze layerData Engineer