31 lines
1.1 KiB
Markdown
31 lines
1.1 KiB
Markdown
|
---
|
||
|
|
created: 2026-01-01
|
||
|
|
updated: 2026-01-01
|
||
|
|
sources: []
|
||
|
|
tags: [llm]
|
||
|
|
---
|
||
|
|
|
||
|
|
# LLM Fusion
|
||
|
|
|
||
|
|
This page documents the concept of model fusion and ensemble techniques.
|
||
|
|
The page is linked from memory-system.md so the orphan check should
|
||
|
|
report zero orphans. The line count is high enough to avoid stub status.
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
Model fusion combines multiple language models into a single inference path
|
||
|
|
or a routing layer that selects among candidate models. The technique is
|
||
|
|
useful for combining specialised models (coding, reasoning, multilingual)
|
||
|
|
without retraining a monolithic base model. The approach is described in
|
||
|
|
detail in the OpenRouter Fusion whitepaper and the DRACO benchmark study.
|
||
|
|
|
||
|
|
## Practical Examples
|
||
|
|
|
||
|
|
OpenRouter Fusion, DRACO benchmark, and similar approaches show that small
|
||
|
|
ensembles of well-chosen open models can match or exceed much larger
|
||
|
|
frontier models on specific tasks. The economic argument is strong: you
|
||
|
|
pay only for inference, not for the full frontier model on every request.
|
||
|
|
The technique is particularly effective when the candidate models have
|
||
|
|
complementary strengths rather than overlapping coverage of the same
|
||
|
|
tasks.
|