Most agent loops end the moment the model decides it's finished. That's the weak link: the model is both the worker and the judge, and a tired judge in a long context will call a half-done deck "complete" and hand it to your customer. Anthropic's Outcomes — a Claude Managed Agents feature in public beta since the May 6 Code with Claude keynote — attacks exactly that. It bolts a scoreboard onto the loop.

If you read one line: Outcomes lets you write a rubric for what "done" means, then runs a separate grader that scores every attempt and sends the work back until it passes — a self-correcting loop instead of a one-shot guess. The mechanism is more interesting than the feature, because you can build most of it yourself.

How it actually works#

You supply a rubric — a markdown document with per-criterion scoring that spells out what a good result looks like. From there the Managed Agents harness runs a tight loop:

  1. The worker attempts the task.
  2. A grader — a separate Claude instance in its own context window — scores the artifact against your rubric.
  3. If it passes, you're done. If it fails, the grader writes a gap analysis: precisely which criteria failed and what's still missing.
  4. That gap analysis is handed back to the worker as a new message, and the worker takes another pass.

Two design decisions make this more than "ask the model to check itself." First, the grader is blind to the trajectory — it never sees the agent's reasoning or the path it took, so it judges the finished artifact rather than the persuasive story of how the artifact was made. Second, the harness withholds the rubric text from the worker on retry and returns only the gap analysis. That's a deliberate anti-gaming move: if the agent could read the rubric, it would learn to satisfy its wording; given only "the executive summary still omits the risk section," it has to fix the actual work. It's a neat structural answer to the reward-hacking problem that plagues self-grading agents.

The numbers, read honestly#

In Anthropic's internal benchmarks, Outcomes lifted task success by up to 10 percentage points over a standard prompting loop, with the largest gains on the hardest tasks — which is where you'd expect a retry loop to pay off, since easy tasks pass on the first try anyway. On file generation the reported quality lift was 8.4% for docx and 10.1% for pptx.

These are vendor numbers on vendor-chosen tasks, so don't bank the exact figures. But the shape is credible and matches what the eval literature has said for a while: a competent verifier plus retries beats a single sample, and the gap is widest on hard, checkable work. It's the same intuition behind best-of-N sampling, with the sampler replaced by a targeted fix-the-gap retry.

What it costs#

There's no free reliability. Every iteration is two model passes — one worker, one grader — so both token spend and latency scale with the number of retries. A task that loops three times before it passes costs roughly three worker runs plus three grader runs. On a cheap, fast, high-volume endpoint that math is brutal; on a report a customer reads once and judges you by, it's trivial.

So the decision is a cost-per-task-times-stakes calculation:

The strategic read#

The quiet story here isn't a product SKU — it's where the reliability lever moved. For two years the answer to "my agent is unreliable" was "wait for a better model." Outcomes says the lever is a loop: a blind grader, a rubric, and a gap-analysis feedback message. That's architecture you own, not weights you rent.

Which is the genuinely useful takeaway for a founder. If you're already on Claude Managed Agents, Outcomes is a header and a rubric away. If you're not, you can build the same worker-grader loop in a few dozen lines against any model — and keep it when you switch providers. We wrote that version up next: how to add a verifier loop to your agent, with the code.