Can you block AI training with one robots.txt line? Google says it will support it
Soon, yes. On 30 October 2026 Google said it will support a shared AI training control in robots.txt, based on work from the IETF AIPREF working group. One policy token, __usage_ai-training, would tell every participating AI provider whether your content may be used for training. Google aims to support it with other providers in 2027, and Google-Extended keeps working alongside it.
- The signal is a normal robots.txt group with a shared “policy token” in the User-agent line:
User-agent: __usage_ai-training. - If you opt out with it and don’t name Google-Extended yourself, Google will treat it as if you had disallowed Google-Extended.
- Company-specific controls such as Google-Extended keep working, so you can set one broad rule and still make exceptions.
- Google’s crawler docs say support is planned for early 2027. The IETF document is still a first draft (-00).
What happened
In a Google Search Central blog post, John Mueller announced plans to support a shared AI training control in robots.txt. Today you need a separate rule for every AI company’s bot. The new signal is meant to work across all AI providers that adopt it. It comes from the IETF draft “Machine-Readable Training Preference Signal”, written in the IETF’s AI Preferences (AIPREF) group, and Google says it is in line with the EU’s Code of Practice for general-purpose AI models.
What this means for your site or store
Nothing breaks today, and nothing switches on until 2027. But the choice gets simpler, so it’s worth deciding now:
- Decide what you want: being found and cited in AI answers is not the same as being used to train AI models. Most businesses want the first. Whether to allow the second is a business choice.
- Know the trade-off with Google: opting out through the shared signal counts as blocking Google-Extended. Google-Extended also controls grounding in Gemini Apps, and Google says sites that allow it can appear in responses and get clicks there. Google Search itself is not affected.
- Review your robots.txt once, not ten times: list the AI rules you have today. When support arrives, one group may replace several of them.
The technical details
The draft adds no new directives. You write an ordinary group, for example User-agent: __usage_ai-training with Disallow: /, and you can still Allow: specific folders. A User-agent: * group does not count as this signal, and the token says how content may be used, not whether it can be crawled. Our guide on allowing AI search bots while blocking AI training has today’s exact lines for GPTBot, Google-Extended and others.
Want to check what your robots.txt tells AI bots right now? Use our free AI-ready toolkit, or get robots.txt, llms.txt and schema done for you with the AI-Ready Kit.
FAQ
Sources
- Supporting a shared AI training control for websites, Google Search Central Blog
- Google’s common crawlers: Google-Extended, Google for Developers
- Machine-Readable Training Preference Signal (draft-kelley-aipref-training-preference-signal-00), IETF
