OpenAI backs a shared AI-training preference signal: what website owners need to know

OpenAI has backed a shared signal through which website owners can express a preference against using their content to train generative AI. The announcement was published on October 30. The aim is to communicate one preference to multiple participating providers instead of maintaining a different set of rules for each. OpenAI plans implementation in the coming months; its existing crawler controls remain in place.

Treat this as news about a forthcoming way to manage content use. It does not mean that every AI company has already stopped collecting website data. For a blog, shop or documentation owner, the important task is deciding which activity to restrict without accidentally blocking useful search traffic at the same time.

What the proposed signal looks like

The Machine-Readable Training Preference Signal draft proposes a special __usage_ai-training value in robots.txt. This minimal example expresses a preference against training for all paths. It illustrates the proposed syntax; it is not a complete replacement for your existing configuration.

User-agent: __usage_ai-training
Disallow: /

As of October 31, this is an individual Internet-Draft, not an approved IETF standard. It separates content-use preferences from crawl rules. Check support with each provider: Google’s documentation currently identifies early 2027 as its intended implementation period for the shared signal.

Training, search and visiting a page are different activities

OpenAI’s crawler documentation already separates GPTBot, which relates to training, from OAI-SearchBot, which serves ChatGPT search. Their settings are independent. A separate ChatGPT-User agent can visit pages following a user action; robots.txt may not apply to those requests.

Your goal	What to check	What not to assume
Restrict AI training	Training crawler rules and support for the new signal	That this automatically stops every page request
Remain available in search	Search crawler access and indexing rules	That every “AI bot” block leaves search unaffected
Protect a private file	Authentication and server access permissions	That robots.txt hides it from people or every crawler

The comparison is a practical decision aid, not a set of interchangeable switches. A shop owner might want an assistant to find an up-to-date product page while opposing its use for training. Write down that desired outcome in ordinary language first, then choose controls for the particular systems involved.

Where Google-Extended fits

Google’s separate Google-Extended token manages content use for future Gemini models and certain grounding uses, where source material supports a response at request time. Google explicitly says this token does not affect inclusion in Google Search and is not a Search ranking signal. It is a different control from blocking Googlebot.

The practical lesson is to avoid assuming that every token has the same effects. If your hosting service offers a “block AI” button, inspect the specific crawler names and rules it adds. The button’s label is too broad to establish the consequences. Save the original configuration so you can compare changes and reverse them when necessary.

Why robots.txt does not replace access protection

As Google Search Central explains, robots.txt relies on crawlers choosing to follow its instructions. It cannot force arbitrary third-party scripts to comply and cannot password-protect a confidential document. A URL disallowed for crawling can also enter an index through links from other pages.

Indexing has a separate control: the noindex directive. Google must retrieve a page to read that instruction, so blocking crawling can prevent it from seeing noindex. If your goal is only to restrict training, do not add noindex to public articles as though it were an equivalent mechanism. That is a different decision with different visibility consequences.

A practical sequence for a small website

1. Write down your goal. Make a short list of what you allow for search, what you allow for training and what should remain private altogether.
2. Locate the current robots.txt. Google documents its location at the root of the relevant host. Save the text and establish whether the website, a plugin or the hosting service produces it.
3. Check actual support. Read current documentation for each provider that matters to you. Announced future support does not replace present controls.
4. Edit the specific rule group. Do not replace the whole file with a short example from an article: you could lose other useful rules or the sitemap location.
5. Check the external result. Open robots.txt without signing into the administration panel. Confirm that the changed text is visible and that the server returns neither an error page nor a CAPTCHA.
6. Inspect important URLs. In Search Console URL Inspection, compare indexed information with a live-page test. Keep the result alongside the date of the change.

If a contractor manages the site, give them the desired policy and ask to see the exact difference between the old and new file. That is easier to assess than a vague statement that “AI has been blocked”. List multiple subdomains separately: a rule on one host does not establish the same configuration on another.

For a small team, also record who owns the decision. An editor might want wider discovery while an administrator is trying to reduce unwanted traffic. Agreeing on the purpose first helps prevent two people from repeatedly overwriting each other’s settings. A short change note should say what changed, why it changed and which service’s documentation supports that choice.

Will this bring a site back into search?

Do not treat support for the new signal as a promise of ranking recovery. URL Inspection exposes parts of a page’s technical state; Google warns that a successful live test does not guarantee indexing or impressions. Traffic losses require their own investigation. Our explanation of how AI uses documents while answering a question can also help separate retrieval from training.

Useful preparation today is to define your policy, keep your current rules and watch for providers’ implementation confirmations. Adding a new entry alone is not evidence that your material has been excluded from every training process. Judge the effect against the documented conditions of each service, rather than the strength of a headline.

Cover: official OpenAI announcement artwork. The original English text is preserved.