hard20 pts · first answer countstransformerslorainference

Merge or keep LoRA adapters separate

You have one base model and 40 customer-specific LoRA adapters. Traffic mixes all customers in each batch. What serving approach fits best?

A.Merge all 40 adapters into the base weights by adding them togetherB.Keep adapters unmerged and use a server that applies the right one per requestC.Deploy 40 full merged copies of the model on separate GPUsD.Retrain one adapter on all customers' data and drop the rest

Discussion 0

Join free to post in the discussion.

Merge or keep LoRA adapters separate

Discussion 0

Related questions