hard20 pts · first answer countsincremental-loadslate-dataidempotency

Rows missed by a strict watermark

An incremental job loads WHERE updated_at > MAX(updated_at) in the target. Replication from the source sometimes commits rows a few minutes late with older updated_at values, and those rows never appear. What is a robust fix?

A.Run the job less oftenB.Re-read a lookback window (for example updated_at > max - 3 hours) and MERGE on the primary key, so overlap is safeC.Use updated_at >= instead of >D.Switch from MERGE to INSERT for speed

Discussion 0

Join free to post in the discussion.

Rows missed by a strict watermark

Discussion 0

Related questions