Applied LLMs
Model Merging: Linear and SLERP
Linear and SLERP merging combine the weight tensors of separately fine-tuned models into a single deployable checkpoint, trading off alignment and capability at zero inference cost.
intermediate · 7 min read · Premium
You can train two models - one specialised for reasoning and one for creative writing - and merge them into a single checkpoint that costs nothing extra at inference time. No additional training, no architectural changes, no ensembling overhead. That is the practical promise of model merging, and it is now routinely used to build competitive open-weight models on Hugging Face.
This concept covers the two most widely used interpolation strategies: plain linear (weighted average) and SLERP (Spherical Linear Interpolation). Both operate entirely in weight space, require no labelled data, and produce a checkpoint identical in shape to either parent.
The geometry of fine-tuned weight space
Keep reading with Pro.
You're reading the preview. Unlock the full concept plus the library, study plans, the AI mentor, and daily emails.