medium10 pts · first answer countsfine-tuninglora

How LoRA is initialised

LoRA initialises A with small random values and B with zeros. Why is B set to zero?

A.So that gradients for A are always zeroB.So the adapter uses less memory at the startC.So the model starts exactly equal to the base modelD.So the learning rate can be ignored

Discussion 0

Join free to post in the discussion.

How LoRA is initialised

Discussion 0

Related questions