Applied LLMs
Composing and Stacking Adapters
Multiple trained adapters can be combined sequentially, by weighted sum, or through attention-based gating to build new capabilities without retraining the base model.
advanced · 7 min read · Premium
Suppose you have already fine-tuned a 7B model with one LoRA adapter for Spanish translation and a second for legal summarisation. A third client needs a Spanish-language legal summariser. The naive answer is to train from scratch. The more interesting answer is to ask whether those two adapters can be combined into something useful without touching the base model again.
That question is what adapter composition is about. It is harder than it sounds, because LoRA and bottleneck adapters are not simply additive in the parameter space, and the ways they can interfere are subtle.
What "composition" actually means
Keep reading with Pro.
You're reading the preview. Unlock the full concept plus the library, study plans, the AI mentor, and daily emails.