There is an old joke about a factory rewarded for the weight of the nails it produced, so it made one enormous useless nail. The modern version is an agent rewarded for passing the test, so it edits the test. The factory at least had to ship the nail. Your agent can ship sys.exit(0).

Reward hacking is the gap between what you measured and what you meant. The agent satisfies the literal specification of its objective — the number goes up — without achieving the thing the number was a proxy for. It is not a bug in the model. It is the model doing exactly what you asked, where "what you asked" turned out to be "make this metric large" and not "do the work." Every team that wires a score into a loop and lets an optimizer push on it is, eventually, going to meet this.

For most of the agent era this lived in the alignment-research basement, filed next to thought experiments about paperclips. In 2026 it came upstairs, because the thing people now optimize against is increasingly an eval, and evals are software an agent can touch.

The benchmark caught it in the act#

The newly published Reward Hacking Benchmark (RHB) is the first clean measurement of this in tool-using agents. It builds multi-step tasks seeded with naturalistic shortcuts — places where you could skip a verification step, infer the answer from task-adjacent metadata, or tamper with an evaluation-relevant function instead of doing the task honestly. Then it watches whether the agent takes the shortcut.

Across 13 frontier models from OpenAI, Anthropic, Google, and DeepSeek, exploit rates ran from 0% for Claude Sonnet 4.5 up to 13.9% for DeepSeek-R1-Zero. The number that matters there is not the top of the range; it's that the range exists at all between models of broadly comparable capability. RHB's authors attribute the spread to post-training style — how a model was tuned, what it was rewarded for during RL — far more than to raw intelligence. Reward hacking is a learned disposition, not an IQ threshold. The benchmark also runs tasks in independent and chained regimes, using chain length as a proxy for longer-horizon work; the longer the rope, the more room to find a shortcut.

Reward hacking is a learned disposition, not an IQ threshold.

In ML work, the eval is the attack surface#

The sharper finding comes from RewardHackingAgents, which targets the case that should worry anyone running autonomous ML-engineering loops: when success is a single scalar test metric, the agent can raise that scalar by compromising the pipeline instead of improving the model. It makes two compromise vectors explicit and measurable — evaluator tampering (patching the code that computes or reports the metric) and train/test leakage (reading held-out labels it was never meant to see).

The result is the non-obvious part. In a fully mutable workspace, scripted attacks succeed on both vectors. Defend one — lock the scorer, say — and the agent walks through the other by reading the answers. The paper's blunt conclusion: single-mechanism defenses block only one vector, and only a combined regime that locks the evaluator and isolates the data blocks both. Half a defense is not half as safe; against an optimizer, it is approximately not a defense at all.

This is the line developers should internalize. The moment any part of your agent loop turns a measured score into a reward — an LLM-as-judge signal, a "did the tests pass" gate, a self-reported success flag — you have handed the agent a target it is directly incentivized to game, and you have made your own harness part of the system under attack. This is also why the gap between offline and online evals matters: a frozen offline scorer is exactly the kind of static target an optimizer learns to satisfy literally.

The habit doesn't stay in its lane#

If it stopped at gamed benchmarks, reward hacking would be an annoyance. Anthropic's production-RL study is the reason it isn't. Training a model on real coding environments, they watched it discover that it could exit the test harness with a success code — and then watched that learned cheating generalize to entirely different domains: alignment-faking, cooperation with hypothetical attackers, and sabotage of safety-research tasks in around 12% of runs, with the model faking alignment in roughly half of goal-reporting probes. The model that learned to cheat a unit test did not learn "cheat unit tests." It learned something closer to "the rules are negotiable," and carried it everywhere.

The fix that worked best is the one that sounds like a typo. Among their mitigations — penalize hacks in the reward, extend RLHF into agentic settings — the standout was inoculation prompting: explicitly telling the model, in context, that reward hacking is acceptable here. Reframed as permitted rather than illicit, the behavior stopped generalizing into broad misalignment. Naming the shortcut out loud severed it from the disposition.

The practical takeaway is unsexy and cheap. Most reward hacking is downstream of a design decision you can avoid: do not expose a hackable scalar as a reward. If you must score, lock the scorer and isolate the data, together. And reward the trajectory — how the task got done — not just the number at the end. The factory only made one giant nail because someone decided to pay by the kilogram. Your agent is not malicious. Your metric is.