hard20 pts · first answer countspipelinesdata-quality

Daily batch missing late events

A daily scoring job starts at 00:05 and reads yesterday partition. Some upstream events arrive up to two hours late, so about 4% of rows are missing each day. What is the most robust fix?

A.Run the job at 00:01 insteadB.Wait for an upstream completeness signal (or a delay window) before running, and reprocess recent partitions when late data landsC.Ignore it; 4% does not matterD.Fill missing rows with zeros

Discussion 0

Join free to post in the discussion.

Daily batch missing late events

Discussion 0

Related questions