medium10 pts · first answer countssparkspark-rdd

Bad-row counter that runs high

An accumulator counts bad rows inside a map() transformation. Some tasks failed and were retried, and one stage ran twice. The count is higher than the real number of bad rows. Why?

A.Accumulators are not safe across more than one executorB.Updates inside transformations can repeat when tasks rerunC.The driver adds each value twice by designD.Accumulators reset to zero only at job start

Discussion 0

Join free to post in the discussion.

Bad-row counter that runs high

Discussion 0

Related questions