The short version — paste this into your robots.txt:
User-agent: __usage_ai-training
Disallow: /
That one group now signals to OpenAI, Google, Microsoft and Anthropic — as of their October 30, 2026 agreement — that they should not use your site to train models. If you want to keep a few paths open (a press kit, say), add Allow: lines under the same group:
User-agent: __usage_ai-training
Disallow: /
Allow: /press-kit/
That's the whole implementation. The rest of this guide is the three things that make it work — and the one mistake that silently voids it.
What this token actually is#
For years, opting out of AI training meant maintaining a separate line for every lab's crawler — GPTBot, ClaudeBot, Google-Extended, and so on — and chasing each new one as it appeared. On October 30, 2026 the four largest labs agreed to honor a single shared token instead: __usage_ai-training.
It's defined in an IETF Internet-Draft, draft-kelley-aipref-training-preference-signal-00, and it is deliberately boring: it introduces no new directives and reuses ordinary robots.txt groups, just with a reserved token in the User-agent line (Unite.AI). That's by design — every CMS, CDN and static host already knows how to serve a robots.txt, so there's nothing new to deploy.
The gotcha that voids it: the wildcard does not count#
Here is the mistake you will otherwise make. You might assume your existing site-wide block —
User-agent: *
Disallow: /
— already opts you out of training. It does not. Under this signal, a User-agent: * group does not count as the training preference. The token __usage_ai-training has to appear by name, in its own group, or the labs read your site as expressing no training preference at all (FourWeekMBA). A wildcard disallow and a missing token are, to a training crawler, two different statements.
So the token group is additive. Keep your * rules for ordinary crawl hygiene; add the __usage_ai-training group on top.
What it controls — and what it doesn't#
Three scope limits decide whether this tool even solves your problem:
- Training, not retrieval. The token asks labs not to train on your content. It does not pull you out of live answer-engine citations — the fetches that happen when someone asks ChatGPT or Perplexity a question and the bot reads your page to answer it. Those run through retrieval bots (
OAI-SearchBotand friends), governed separately. If your goal is "keep answering engines citing me but stop them training on me," this token is exactly right. If your goal is "disappear from AI answers entirely," it's the wrong lever. - Forward-looking, not retroactive. It does nothing about content already collected into datasets or already trained into shipped models. You're setting policy for future training runs, not clawing anything back.
- Voluntary. Like everything in
robots.txt, it's a machine-readable statement of intent, not a technical block. The four labs have committed to honor it — but the signal's force comes from their compliance, not from enforcement. Rollout is staggered: Anthropic has begun, OpenAI says "the coming months," Microsoft names February 1, 2027, and Google says 2027 (Relevant Audience).
Keep your per-company controls too#
The shared token doesn't retire the single-vendor controls — it layers on top of them. If you want to also block a specific vendor's crawler from fetching at all, or distinguish training from grounding for one company, keep those lines. Google still reads Google-Extended; OpenAI still reads GPTBot and OAI-SearchBot; Anthropic still reads ClaudeBot. A reasonable 2026 robots.txt has both: the cross-vendor training token for the broad signal, plus any per-bot rules where you need finer control.
One caution against copy-paste confusion: this token is not the same as Cloudflare-style Content-Signal lines (ai-train=no) or the older Usage-Pref HTTP-header draft. Those are separate conventions. The thing the four labs agreed to on October 30 is specifically the __usage_ai-training user-agent token in robots.txt — there is no official HTTP-header or <meta> variant of this signal today.
Verify it's live#
Deploy, then confirm the file actually serves the group:
curl -s https://yourdomain.com/robots.txt | grep -A2 __usage_ai-training
You should see your token group echoed back. If your site is behind a CDN or a framework that generates robots.txt, make sure the token survives the build — a static public/robots.txt is the least surprising place to put it.
The honest bottom line#
This is the cleanest opt-out mechanism the open web has had: one line, four labs, standard syntax. It is also only as strong as the labs' word, it only touches training, and it starts from today — not from everything already ingested. For a founder with a content moat, that's still worth ten minutes: add the group, keep your per-bot rules, and check it with curl. For the full context on why all four labs moved at once, see this morning's Wire.



