The single cheapest reliability upgrade for an agent isn't a bigger model — it's a second one that grades the first. Anthropic just shipped this as a managed feature called Outcomes: a rubric-graded loop that iterates until the work is good enough. The good news for anyone not on Managed Agents is that the mechanism is simple, portable, and about forty lines of code. Here's how to build it yourself.

If you read one line: a verifier loop is worker produces → separate grader scores against a rubric → feed the gap back → repeat until it passes or the budget runs out. Two rules make it work: grade in a fresh context, and feed back the gap, not the rubric.

The shape#

                ┌─────────── gap analysis ───────────┐
                ▼                                     │
   ┌────────┐        ┌────────┐        ┌─────────────────┐
   │ worker │──────▶ │ artifact│──────▶│ grader (rubric) │
   └────────┘        └────────┘        └─────────────────┘
                                              │  pass?
                                              ▼
                                            return

The worker doesn't decide when it's done — the grader does, against criteria you wrote in advance. That's the whole idea. Everything below is the details that make it reliable instead of theater.

The code#

We'll use the Claude API, but the pattern is provider-agnostic. First, the worker — an ordinary generation call:

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-opus-4-8"

def run_worker(task: str, feedback: str | None = None) -> str:
    """Produce (or revise) the artifact."""
    prompt = task if feedback is None else (
        f"{task}\n\nYour previous attempt fell short. Fix exactly this and "
        f"return the full corrected result:\n{feedback}"
    )
    msg = client.messages.create(
        model=MODEL,
        max_tokens=8000,
        messages=[{"role": "user", "content": prompt}],
    )
    return next(b.text for b in msg.content if b.type == "text")

Now the grader. It runs in a fresh call — a new context that never sees the worker's reasoning — and returns a structured verdict via structured outputs, so you never parse prose:

from pydantic import BaseModel

class Verdict(BaseModel):
    passed: bool
    gaps: list[str]   # empty when passed; specific and actionable when not

def grade(artifact: str, rubric: str) -> Verdict:
    """Score the artifact against the rubric, blind to how it was made."""
    result = client.messages.parse(
        model=MODEL,
        max_tokens=2000,
        output_config={"effort": "high"},  # judging is worth the extra thinking
        messages=[{
            "role": "user",
            "content": (
                "You are a strict grader. Score ONLY the artifact below against "
                "the rubric. Judge the finished work, not any explanation of it. "
                "For each failed criterion, state precisely what is wrong.\n\n"
                f"# Rubric\n{rubric}\n\n# Artifact\n{artifact}"
            ),
        }],
        output_format=Verdict,
    )
    return result.parsed_output

And the loop that ties them together — with a retry budget and an honest terminal state:

def solve(task: str, rubric: str, max_iters: int = 4) -> dict:
    feedback = None
    for attempt in range(1, max_iters + 1):
        artifact = run_worker(task, feedback)
        verdict = grade(artifact, rubric)
        if verdict.passed:
            return {"ok": True, "artifact": artifact, "attempts": attempt}
        # Feed back the GAP, not the rubric — see below.
        feedback = "\n".join(f"- {g}" for g in verdict.gaps)
    return {"ok": False, "artifact": artifact, "attempts": max_iters,
            "unmet": verdict.gaps}   # a real outcome, not a swallowed error

That's the entire loop. Point it at a task and a rubric:

res = solve(
    task="Extract every line item from this invoice as JSON with a numeric "
         "`total`. Invoice text: ...",
    rubric=(
        "1. Output is valid JSON, no prose around it.\n"
        "2. There is a top-level numeric `total` field.\n"
        "3. Every line item has `description` (string) and `amount` (number).\n"
        "4. The line-item amounts sum to `total` within 0.01."
    ),
)

The two rules that make it work#

1. Grade in a fresh context. Notice grade() starts a brand-new call — it never receives the worker's messages, drafts, or reasoning. This is not an optimization; it's the point. A model asked to check its own work in the same conversation has already committed to an answer and will defend it. A blind grader has nothing to defend, so it judges the artifact. This is exactly the design Anthropic uses in managed Outcomes, where the grader "evaluates the artifact in its own context window, uninfluenced by the agent's reasoning."

2. Feed back the gap, not the rubric. The loop passes verdict.gaps — "the amounts don't sum to total" — back to the worker, never the rubric text itself. Show the model the rubric and it learns to satisfy its wording; give it the specific defect and it has to fix the work. Skip this and you get reward hacking: outputs that score well and are still wrong.

Write rubrics that can actually be graded#

The loop is only as good as the rubric. Every criterion should be something a careful reader could mark pass/fail without a judgment call:

Vague criteria produce a grader that flip-flops between attempts, and the loop never converges — it just burns your retry budget. If you genuinely can't write checkable criteria, a verifier loop is the wrong tool; you want a human in the loop or a different approach. (A useful trick: hand a known-good example to the model and ask it to reverse-engineer the rubric that example satisfies.)

When to reach for it#

Each iteration adds one grader call, so a task that loops three times costs roughly three worker runs plus three grader runs. That math is trivial on a report a customer reads once and brutal on a cheap high-volume endpoint. Use the loop where "wrong" is expensive and "done" is checkable — structured extraction, generated documents and decks, data migrations, anything a customer judges you by. Skip it for fast, cheap, or subjective work.

This is the build-it-yourself version of the scored agent loop. For the managed feature it mirrors — and why the reliability lever moving from the model into a loop is the more important story — see The Agent Loop Gets a Scoreboard. And if you're weighing this against just sampling the worker several times and voting, that tradeoff is best-of-N vs. self-consistency — a verifier loop wins when you can check the answer, sampling wins when you can only compare answers to each other.