E6 · Mixture-of-agents (layered)

Principle

Several LLMs (the proposers) answer the same request. The next layer reads their answers and produces better ones, layer after layer. A final aggregator synthesises everything.

The problem it solves

A single model, even a large one, hits a ceiling on some tasks; combining several models can beat the best of them.

When to use it

• You aim for the best possible quality on an open-ended task, latency matters little, and several complementary models are available.

When to avoid it

• The use is interactive: layers run one after another, so the first token arrives late.
• The budget is tight: cost is multiplied by the number of models and layers.
• Gains are mainly measured on preference benchmarks judged by an LLM: check on your own tasks before adopting.

Common pitfalls

The anti-patterns that misuse this pattern, best established first. Each sheet describes the symptoms, the public incidents and the remedies.

Loop, stopping and orchestration

• AP41 · Conformist debate — Agents debate to correct one another and end up agreeing on the most confident answer, even a wrong one.

All anti-patterns

Synonyms by vendor

• Mixture-of-Agents (MoA) (Wang et al. (ICLR 2025))

Neighbouring patterns

• E5 · Group chat and debate
• E1 · Orchestrator and subagents
• B4 · Sampling and voting

Sources

• Mixture-of-Agents Enhances Large Language Model Capabilities — Wang et al., ICLR 2025, arXiv, 2024-06