8. Creator-Verifier Pattern
The creator-verifier pattern assigns production and evaluation to separate steps or agents. It is a useful design candidate, not a guaranteed accuracy multiplier.
The evidence is mixed. Self-Refine reports gains from iterative same-model feedback across seven tasks. Multiagent Debate reports improvements on selected reasoning and factuality tasks. Conversely, Large Language Models Cannot Self-Correct Reasoning Yet finds that intrinsic self-correction without external feedback can fail or degrade answers. Scalable oversight experiments find task-dependent rather than universal gains, and LLM-as-a-judge bias research documents systematic evaluator biases. Anthropic's evaluator-optimizer guidance therefore recommends the pattern when evaluation criteria are clear and iterative refinement produces measurable value.
A fresh context is only one dimension of independence. A verifier can still share the creator's blind spots when both depend on the same model family, specification, tools, retrieval corpus, reward signal, or missing runtime evidence. Assess at least five dimensions:
| Independence dimension | Failure when shared | Stronger design |
|---|---|---|
| Context | The reviewer inherits the creator's framing and assumptions | Give the reviewer requirements, artifact, and evidence without the creator's chain of reasoning |
| Model or provider | Correlated blind spots and self-preference | Stratify results by model and test provider diversity where risk justifies the cost |
| Evidence and tools | Both agents inspect the same incomplete signals | Add deterministic tests, runtime traces, external sources, or a different inspection tool |
| Role and incentives | The reviewer optimizes for agreement or style | Require per-requirement pass/fail verdicts and evidence, not a generic quality score |
| Escalation authority | A probabilistic verdict silently becomes final | Define appeal, timeout, human checkpoint, and fail-closed rules for high-impact decisions |
Measure rescued failures, false accepts, false rejects, regressions, latency, and cost. Counting reviewer calls or using a different role name does not establish independence.
Cross-session messaging can carry the handoff between independently launched Claude Code sessions, but the transport does not make the verifier independent. Bind the handoff to a commit SHA, send the original requirements, artifact, evidence, and uncertainty instead of the creator's rationale, then let the verifier inspect its own current state. Apply the cross-session threat model separately for sender trust, inbound policy, permissions, and cross-machine exposure.
Liza implements this pattern as doer/reviewer pairs backed by a deterministic supervisor. Its state machine and merge authority can reject invalid transitions even when an agent proposes them. That is stronger than a role name in a prompt, but it still needs outcome evaluation: a structurally separate reviewer can share the same model, incomplete specification, or missing evidence as the doer.
The pinned Liza configuration makes the boundary visible. Some high-impact planning stages require a quorum of two and mark provider diversity as preferred, while the coding pair uses a quorum of one. Liza's own architectural issues ledger states that provider diversity is not enforced at verdict submission, reviewer accuracy is not yet measured, and human checkpoints remain load-bearing. The code enforces workflow legality; it does not prove that a legal reviewer verdict is correct.
Judgment allocation: where responsibility actually sits
A stopping condition answers when a loop ends. It does not by itself establish that the result is good enough. Every production design should record who sets the quality bar, what evidence is admissible, who decides that the evidence is sufficient, and who handles exceptions.
| Decision | Default owner | Evidence | Escalate when |
|---|---|---|---|
| Product intent and acceptable risk | Accountable human or product authority | Requirements, policy, risk classification | Intent is ambiguous or impact is high |
| Task decomposition and routing | Orchestrator or planning agent | Typed task graph, dependencies, budgets | Fan-out is large, irreversible, or crosses domains |
| Execution | Runtime and repository harness | Tool results, sandbox events, changed artifacts | A permission, policy, or environmental boundary is reached |
| Mechanical acceptance | Deterministic gate | Tests, schemas, linters, policy checks, reproducible commands | Coverage is incomplete or a check is flaky |
| Semantic sufficiency | Reviewer agent for bounded work, accountable human for high-impact work | Requirement-level verdicts plus inspected evidence | The verifier is uncertain, conflicted, or outside its evidence boundary |
| Release or policy exception | Human or external policy authority | Signed approval and auditable rationale | Never silently auto-approve an exception |
This is the useful insight from the loop-engineering debate: human judgment does not necessarily disappear when prompts are automated. It moves to loop design, quality criteria, exception policy, and final accountability. Own the Outer Loop presents this as a practitioner design principle. Treat the exact placement as a risk decision, not a universal rule.
Practical implementation: spawn a second agent with the output artifact and the original requirements. Ask whether the output satisfies each requirement. Do not ask "is this good?" Ask "does this satisfy requirement X?" with explicit pass/fail for each.
An executable-spec variant makes that contract more granular:
- A planner extracts explicit requirements and acceptance evidence from the task.
- One verifier evaluates each requirement independently, in parallel where isolation allows it.
- Each verifier inspects the relevant codebase state, not only the produced diff.
- Runtime behavior is checked in an ephemeral browser or sandbox when static inspection cannot prove it.
- The harness aggregates per-requirement verdicts and preserves the evidence behind each one.
Shachar Azriel presents this architecture in Executable Specs: Building a Verification Layer for Agentic Coding. The talk documents an implementation approach. It does not independently prove the resulting product's accuracy, so treat the pattern as a design candidate and test it against real repository tasks.
This does not eliminate hallucination; it catches the subset of hallucinations that are inconsistent with the stated requirements. For catching hallucinations that are internally consistent but factually wrong, you need domain-specific test cases.