Skip to content
RED SAGSkills
medium10 pts · first answer countstransformersfine-tuninglora

LoRA parameters for one 4,096 matrix

You add a rank-8 LoRA adapter to one 4,096 × 4,096 weight matrix. How many trainable parameters does that adapter add?

Choose one answer

Discussion 0

Join free to post in the discussion.

Related questions

More Transformers and LLM internals questionsMore Fine-tuning questionsMore Lora questions