medium10 pts · first answer countslorafine-tuning
Trainable parameters in a LoRA adapter
You add a LoRA adapter of rank 8 to one 4096 x 4096 weight matrix. How many trainable parameters does the adapter add?
Discussion 0
Join free to post in the discussion.
Related questions
- Fine-tuning a CNN with batch size 2Machine Learning Engineer
- How DPO differs from PPO-based RLHFMachine Learning Engineer
- Fine-tuned model forgets general skillsMachine Learning Engineer
- Knowledge that changes every dayAI Engineer (LLM & Generative AI)
- What LoRA trainsAI Engineer (LLM & Generative AI)
- Counting LoRA parametersAI Engineer (LLM & Generative AI)
- Fine-tuned model scores too wellAI Engineer (LLM & Generative AI)
- Serving many fine-tuned variantsMLOps Engineer