medium10 pts · first answer countskafkaidempotencydelivery-semantics

Duplicates from at-least-once delivery

A consumer has at-least-once delivery from Kafka and writes to a warehouse. After a restart you see some duplicate rows. What is the right fix?

A.Switch to at-most-once so nothing is ever repeatedB.Reduce the number of partitions to 1C.Delete duplicates by hand each morningD.Make the write idempotent, for example MERGE on a unique event_id instead of a plain INSERT

Discussion 0

Join free to post in the discussion.

Duplicates from at-least-once delivery

Discussion 0

Related questions