For most of the last year, the received upgrade path for instructing a coding agent went one direction: a plain AGENTS.md file was the crude starting point, and Agent Skills — folders of instructions the model loads on demand, with progressive disclosure so you don't blow the context window — were the grown-up version. Skills were the answer to "your context file doesn't scale." Then Vercel ran the head-to-head, and the crude starting point won outright.

The numbers are stark enough that they've been flattened into a meme. A compressed 8KB docs index embedded directly in AGENTS.md hit a 100% pass rate across Vercel's 19 Next.js-specific agent evals — each scored on whether the agent's output actually built, linted, and passed tests, with retries to rule out model variance. The Skill-based approach maxed out at 79%, and it did that in the configuration where the prompt explicitly told the agent to use the Skill. The headlines wrote themselves: a dumb markdown file beats fancy Skills.

That reading is wrong in a way worth fixing, because the wrong lesson tells you to rip out progressive disclosure, and progressive disclosure isn't what failed.

The winner is also progressive disclosure#

Look at what Vercel actually put in AGENTS.md. Not the docs. An index — 8KB, compressed down from a 40KB full dump, roughly an 80% cut — that tells the agent which version-matched documentation files to read for the Next.js version in play, and lets it pull those files as needed. The agent still reads reference material on demand. It still doesn't carry 40KB of docs in every turn. That is the exact mechanism Skills are sold on.

So the comparison was never static-file-versus-smart-retrieval. Both sides do retrieval. The single variable that moved was where the retrieval trigger lives — and whether the model can skip it.

A Skill's index loads only if the agent decides to invoke it. An AGENTS.md index is in the system prompt whether the agent decides anything or not. That's the whole delta.

The failure data makes this concrete. Vercel found the Skill simply wasn't invoked in 56% of runs — more than half the time, the agent never opened the door. The Skill's contents were fine. The problem is that a Skill introduces a decision point: the model has to notice it needs the Skill, choose to call it, and do so before it's already committed to a wrong approach. Each of those is a coin flip you added to the pipeline. An always-present index removes all three at once, because there's nothing to decide.

Brittleness hides in the wording#

The part that should worry anyone shipping Skills is how sensitive the invocation was to phrasing. Vercel reports that changing how the agent was told to use the Skill — "invoke the Skill first" versus "explore the project first" — flipped outcomes on identical content. In one eval the "invoke first" framing produced a correct page.tsx but missed the required next.config.ts changes entirely; the "explore first" framing got both. Same Skill, same docs, different trigger sentence, different result.

That's the tax nobody prices in. A Skill isn't just its content; it's its content plus a fragile little social contract about when the model should reach for it, and that contract is written in the same natural language the model is free to reinterpret. You are, in effect, prompt-engineering the retrieval decision — and as with any prompt, small wording changes carry real variance. Static context has no such contract. It's just there.

What this does and doesn't prove#

Be honest about the scope, because the meme isn't. This is a coding-agent result, on one framework's documentation, run by the company that ships that framework. It's directional evidence, not a universal verdict, and it lands hardest exactly where it was measured: feeding an agent the conventions of a codebase it's editing. Vercel's own next.js issue tracker even logged that the compressed index wasn't reliably regenerating for some setups — the technique has sharp edges too.

Skills still earn their keep. When a capability is large, self-contained, and needed only occasionally — a whole PDF-generation toolchain, a niche migration playbook — you don't want it burning tokens in every turn, and the invocation cost is worth paying for the context you save. That is the case Skills were designed for, and OpenAI's guidance to test Skills with evals exists precisely because their reliability has to be measured, not assumed. The mistake is using a Skill for conventions that must apply every single time — your lint rules, your file layout, your framework version's gotchas. Those aren't occasional. Making their entry point optional is the bug.

The design rule#

Strip away the horse race and Vercel's result reduces to one line you can apply without their eval harness: put must-always-apply knowledge where the model can't skip it, and reserve on-demand loading for what you can afford to have the model miss.

AGENTS.md wins the coding case not because it's simpler but because "always in context" is a stronger guarantee than "invoked when the model feels like it" — and for a codebase's non-negotiable conventions, you want the guarantee. Progressive disclosure is still the right idea for keeping the window lean; you just don't hang your non-negotiables off a trigger the agent gets to ignore. The 56% is the number to remember. More than half the time, the smart system chose not to be smart — and the dumb one didn't have to choose at all.