hard20 pts · first answer countssparkspark-mllib

Rows in both train and test

train, test = df.randomSplit([0.8, 0.2], seed=7) is used on a DataFrame built from a non-deterministic source and not cached. Some rows appear in both splits. Why can this happen?

A.randomSplit samples with replacement by defaultB.Each split recomputes df, and the rows or order can differC.A fixed seed makes overlap certain on purposeD.Spark adds 10% overlap to balance the splits

Discussion 0

Join free to post in the discussion.

Rows in both train and test

Discussion 0

Related questions