medium10 pts · first answer countsgpu-computingdistributed-training

DDP error about unused parameters

DDP raises an error because some parameters received no gradient this step (for example a branch was skipped). Setting find_unused_parameters=True fixes it. What is the cost?

A.All parameters are frozen permanentlyB.Gradients are no longer averagedC.An extra graph walk every step, slowing trainingD.The model must now run on just one GPU

Discussion 0

Join free to post in the discussion.

DDP error about unused parameters

Discussion 0

Related questions