Google, OpenAI, Microsoft and Anthropic back a shared robots.txt signal for opting out of AI training
Google will support a shared AI training control in robots.txt based on an IETF AIPREF draft: one 'policy token' states a site's training preference to many AI providers at once, while controls like Google-Extended keep working. The Verge reports OpenAI, Microsoft and Anthropic will honor it too; Anthropic already does, Google and Microsoft plan support next year, OpenAI in coming months.
Why it matters
It is the first time the major AI labs have lined up behind one opt-out signal for training, simplifying crawler control for publishers and shaping how training data is gathered.
Sources3
Google Search CentralOfficial · BlogSupporting a shared AI training control for websitesView original · The VergeMediaThe biggest names in AI are backing a new way to opt out of scrapingView original ·
Microsoft Bing BlogMediaYour content, your choice: clearer preferences for training and groundingView original ·
Written by dalao.ai: the title, summary and why it matters are our own words. The original reports are the record; their publishers keep authorship.