FreeLLMOpenAI to honor robots.txt signals for AI training preferences
OpenAI will respect a shared robots.txt signal, letting website owners specify if their content can be used for AI model training.
OpenAI announced a new policy to honor a shared robots.txt signal for AI training.
The signal will allow website operators to communicate their preferences about whether their data may be used by participating AI providers.
OpenAI says it will follow the signal when it is present, aiming to give creators more control over their content.
The move follows growing calls for clearer data usage rules in the AI industry.
OpenAI did not specify a rollout date or which providers will be included beyond its own services.
What it means for you
For AI users, the policy could reduce the amount of web data that models are trained on, potentially affecting model breadth. For developers of free or local models, it underscores the importance of respecting site owners' signals when sourcing training data.
Sources
- Making it easier for websites to express AI training preferences, OpenAI (official), 30 Oct 2026
Written automatically from the sources above, then checked by our code: every number and name must appear in them, and no sentence may copy a source. Spotted a mistake? Report it.
Which AI can you use for free today?
Every free AI model, ranked by blind human votes, with the memory and speed it needs on your own GPU.