hard20 pts · first answer countsdata-pipelinesschema-evolutiondata-contracts
A new CSV column shifts the data
A partner adds a discount column in the middle of a daily CSV. Your loader maps columns by position. The load succeeds, but amounts now hold discounts. What makes the pipeline robust to this?
Discussion 0
Join free to post in the discussion.
Related questions
- ETL vs ELTData Engineer
- Idempotent pipelinesData Engineer
- Batch or streaming for a daily reportData Engineer
- Incremental logic for a tiny tableData Engineer
- What SCD Type 1 doesData Engineer
- Keeping the old customer segmentData Engineer
- Why a dimension has its own keyData Engineer
- The bronze layerData Engineer