medium10 pts · first answer countsllm-servinglora

Serving many fine-tuned variants

Thirty customers each have a LoRA fine-tune of the same base LLM. What is the most cost-effective way to serve them?

A.Merge each adapter and run 30 full model copies on separate GPUsB.Retrain one model on all customers dataC.Load the base model once and apply the right small adapter per requestD.Serve only the base model and ignore the adapters

Discussion 0

Join free to post in the discussion.

Serving many fine-tuned variants

Discussion 0

Related questions