medium10 pts · first answer countstransformersfine-tuninglora

What QLoRA quantises

QLoRA lets people fine-tune large models on a single GPU. What does it do?

A.Quantises the LoRA adapters to 1 bit while the base stays in fp32B.Trains all weights in 4-bit, including the optimiser statesC.Removes half of the layers and fine-tunes the rest with LoRAD.Keeps the frozen base in 4-bit and trains LoRA adapters in higher precision

Discussion 0

Join free to post in the discussion.

What QLoRA quantises

Discussion 0

Related questions