B5 · Generator-evaluator

Principle

A first LLM generates an answer. A second one evaluates it against criteria written in advance and sends back precise feedback. The loop goes on until the evaluator accepts, or an iteration cap is reached.

The problem it solves

One-pass generation is sometimes unreliable, and a model judging its own work tends to find it good.

When to use it

• Evaluation criteria are clear and refinement brings measurable value: literary translation, code with tests, a document held to a style guide.
• The evaluator has an external signal: tests, a linter, an application driven in a browser.

When to avoid it

• The criteria are vague: the verifier approves without examining, or the loop oscillates.
• The same agent rereads its work with no verification tool: separate the evaluator, equip it, and set an iteration cap.

Common pitfalls

The anti-patterns that misuse this pattern, best established first. Each sheet describes the symptoms, the public incidents and the remedies.

Loop, stopping and orchestration

• AP45 · Multi-hop latency — LLM calls chained in series with no latency budget: the answer arrives after the sum of every step.

Verification and evaluation

• AP56 · Shipping without a verification oracle — The references look real and the tests have “passed”: nobody checked either.
• AP49 · Self-evaluation — The same agent produces and grades: it almost always finds its own work good.
• AP50 · Agent-writable oracle — The agent can see or rewrite what grades it, and ends up optimising the grade rather than the task.
• AP48 · Unvalidated LLM judge — An LLM grades the outputs, and nobody has checked that it grades like a human expert.
• AP51 · Green tests as the only completion criterion — The test suite passes, so the job is done: except the review rejects the PR.

Oversight, operations and governance

• AP77 · Silent regressions on change — The model, the prompt or the infrastructure changes, and only users notice the drop in quality.

All anti-patterns

Synonyms by vendor

• Evaluator-optimizer (Anthropic (2024))
• Generator-verifier (Anthropic (2026))
• Maker-checker / critic loops / reflection loops (Microsoft)
• Review and critique / Iterative refinement / Loop (Google Cloud)
• Generator-critic (« le bureau du rédacteur en chef ») (Google ADK)
• Evaluators and reflect-refine loops (AWS)
• Cross-reflection (CSIRO Data61)
• Reflection à deux agents (A. Ng)

Neighbouring patterns

• C3 · Reflection
• B4 · Sampling and voting
• B6 · Custom state graph

Sources

• Harness design for long-running application development — Anthropic (P. Rajasekaran), 2026-03-24
• Building multi-agent systems: when and how to use them — Anthropic (C. Phillips), 2026-01-23
• Multi-Agents: What’s Actually Working — Cognition (W. Yan), 2026-04-22
• Why Do Multi-Agent LLM Systems Fail? — M. Cemri et al., 2025