Model Merging
A model is just a giant list of numbers, and sometimes you can combine two models by literally averaging those numbers — no training, no GPUs, plain arithmetic on weight files. It works only when both models were fine-tuned from the same base: their weights then sit close together in the same landscape, and interpolating between them lands in a region that keeps much of both specialties. This is why open-weight communities are full of coder-writer-roleplay hybrid merges. Averaging two unrelated models fails completely — their weights have no shared coordinate system to interpolate in.
Drag the slider from pure coder to pure writer. The merged weights are a weighted average, and the ability bars interpolate — note the shared base knowledge holds steady regardless of the mix.
Both models were fine-tuned from the same base, so their weights live close together and averaging them lands in a valley that keeps both skills partially. No training run needed: merging is plain arithmetic on weight files. Merge two unrelated models and this breaks completely.
Check yourself
Averaging two fine-tunes of the same base works, but averaging two models trained from different random starts produces garbage. What do the two fine-tunes share that makes their average meaningful?
Go deeper (free): Hugging Face — Merge LLMs with mergekit ↗