Your coding agent passes the interview and fails the job. It scores in the mid-to-high 70s on SWE-bench Verified — localize the bug, write the patch, ship the pull request — and in a demo it looks like a senior engineer. Then you hand it a real feature, the kind that touches eight files and takes an afternoon, and somewhere in the middle it declares the code done when it isn't, or announces the task is impossible, or simply stops.
SWE-Marathon is the benchmark built to measure exactly that failure. Where SWE-bench asks for one fix, SWE-Marathon asks for a whole project: 20 ultra-long-horizon tasks that a frontier agent burns an average of 27.2 million tokens attempting. And the scores are brutal in a way that should reset how you read every other leaderboard.
The number that matters: 42, not 77#
On SWE-bench Verified, the top coding models cluster in the mid-to-high 70s. On SWE-Marathon, the entire leaderboard tops out at 42.0 — and the model holding that top spot is Kimi K3, Moonshot AI's open-weight release, ahead of Claude Opus 4.8 (40.0), GPT-5.6 Sol (39.0), and Fable 5 (35.0).
The gap between 77 and 42 is not noise. It is the difference between "can fix a ticket" and "can finish a project" — and it is the gap your agent falls into every time it stalls on real work.
Don't read 42 as failure. This is one of the hardest agentic-coding benchmarks in existence; a 42 is the current high-water mark, not a weak grade. The point of the low absolute numbers is the spread they compress: on single-patch work the frontier looks solved, and on sustained work every model — open or closed — fails the majority of tasks. The demo and the job are different benchmarks, and almost nobody was measuring the job.
What actually breaks#
The most useful thing SWE-Marathon publishes isn't the ranking. It's the taxonomy of why agents fail, and every founder who has watched an agent work will recognize all three:
- Poor self-verification. The agent runs, looks at its own output, and concludes broken code is working. It can't reliably tell "done" from "looks done."
- Self-reported infeasibility. It decides the task can't be completed and quits — sometimes on tasks that are demonstrably solvable.
- Premature termination. It ends the run with real work still on the table, having convinced itself the job is finished.
Notice what these have in common: none of them is a failure of coding skill. They're failures of judgment about state — knowing where you are in a long task, whether the thing you built actually works, and whether you're allowed to stop. That's the muscle a short benchmark never exercises, because a single-patch task is over before judgment about the long game matters.
Why you can't game it#
A benchmark this influential invites gaming, so SWE-Marathon is built to resist it. The tests that grade a solution are hidden and written after the fact, so an agent can't optimize against a visible test suite. And a replay mechanism re-runs attempts to catch a model that has memorized a known fix rather than reasoning its way to one. When a training set has quietly absorbed half the public benchmarks — the reason SWE-bench Verified scores keep creeping up — post-hoc hidden tests are how you keep measuring reasoning instead of recall.
The open-weight footnote that isn't a footnote#
Kimi K3 leading this board is the quiet story. On the hardest, longest coding work — the work that most resembles owning a feature — the top of the leaderboard is an open-weight model you can download and self-host, edging out three closed flagships. For a founder deciding where to spend on a coding backend, that reframes the trade: the premium you pay for a closed frontier model buys you a lead on short tasks that evaporates on long ones. (We keep a running scorecard of where open beats closed, and a head-to-head on Opus 5 vs Kimi K3 for agentic coding.)
What to do Monday#
The practical takeaway is one sentence: stop using SWE-bench score as your proxy for "can this agent own a feature." A high Verified score predicts good single-patch work and nothing about long-horizon completion.
So scope for the failure the benchmark just quantified. Slice agent work into pieces short enough that premature termination costs you a paragraph, not a day. Put a real verification gate between the agent and "done," because it cannot be trusted to grade itself. Build explicit hand-back points where a human checks state before the agent commits to the hard middle. This is the same lesson our postmortems keep landing on about why agents fail in production — SWE-Marathon just gave it a number.
The benchmarks that made coding agents look finished were measuring the sprint. SWE-Marathon measures the marathon, and the results say plainly: no model has run one yet. The one in front is open — and it's still, mostly, losing.
Related: how the short-task benchmarks differ — SWE-bench Pro vs SWE-bench Verified and Terminal-Bench vs SWE-bench — and the other long-horizon entrant, SWE-EVO vs SWE-bench.
