Does blocking GPTBot remove you from ChatGPT search?

No. GPTBot governs training. ChatGPT's search feature runs on a completely separate token called OAI-SearchBot, and the two are configured independently. OpenAI states it in one sentence in its own bot documentation: "a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot" (OpenAI, retrieved 2026-08-15).

If you added User-agent: GPTBot / Disallow: / last year to keep your writing out of a model, you did not remove yourself from ChatGPT. You are still eligible to be cited there. That is usually what people wanted, and most of them do not know they already have it.

The short answer

• GPTBot decides whether your pages feed OpenAI's foundation models.
• OAI-SearchBot decides whether you show up in ChatGPT search answers.
• Blocking GPTBot has no effect on OAI-SearchBot, because robots.txt groups are matched per user agent.
• The block that actually costs you visibility is a Disallow on OAI-SearchBot, and it buys you no training protection at all.

The three OpenAI tokens and what each one governs

OpenAI documents four user agents. Three of them decide whether your content is read, and one is about ads. Here is what each one does, quoted from the operator rather than paraphrased.

The row that matters is the second one. OAI-SearchBot is not a training crawler, so a Disallow on it is a pure loss. You give up citations in ChatGPT and get no training protection in return.

The robots.txt that allows search and refuses training

This is the configuration most people describe when you ask what they want. It keeps you out of the training set and keeps you eligible to be cited in ChatGPT.

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

Three separate groups, not one. The robots.txt spec matches a crawler against the most specific group that names it, so a rule written under GPTBot is invisible to OAI-SearchBot. There is no way to express "block training, allow search" in a single group, and any snippet that tries is doing something else.

The Allow: / lines are technically redundant when your file already has a permissive User-agent: * group. Write them anyway. A named group is documentation for the next person who opens the file. It also stops a future "block all AI bots" edit from quietly catching the token you meant to keep.

ChatGPT-User is a different decision from the other two

ChatGPT-User is not a crawler. OpenAI describes it as handling "certain user actions in ChatGPT and Custom GPTs" and says it "is not used for crawling the web in an automatic fashion". It also states plainly that ChatGPT-User "is not used to determine whether content may appear in Search".

So blocking ChatGPT-User does not protect you from training and does not change your search eligibility. It only breaks the case where a real person hands ChatGPT your link and asks about it. That is a strange thing to want.

There is one honest caveat, and OpenAI publishes it: "Because these actions are initiated by a user, robots.txt rules may not apply." Your file expresses a preference here. It is not a guarantee, and no robots.txt directive ever was. If you need content actually withheld, robots.txt is the wrong tool and always has been.

Your access log settles the argument, robots.txt only starts it

A user agent string is a claim, not an identity. Anyone can send GPTBot in a header, and plenty of scrapers do. OpenAI publishes the IP ranges for each bot separately, which is the part of this that is actually verifiable.

We pulled all three on 2026-08-15:

Three files, three disjoint sets of ranges, which is the same separation the robots.txt tokens describe, expressed in infrastructure. Match your access log against them and you can see which of the three jobs OpenAI was doing on any given request, and whether a robots.txt change altered anything at all.

Run that check once. The gap between "I edited robots.txt" and "the fetches changed" is where crawler advice quietly fails, and a grep over a week of logs closes it faster than any tool will.

What our own robots.txt was doing without us

Ours is a useful example because we did not plan it, and because writing this post is what made us look.

On 2026-08-15, https://seobuilder.tech/robots.txt opened with a block of rules that were nowhere in this codebase. src/app/robots.ts generates eight Disallow lines for authenticated and transactional paths and a sitemap reference, and nothing about AI at all. Everything above that was injected at the edge by Cloudflare's Managed robots.txt feature, which writes a fixed list of AI crawler tokens into the served file. The file on localhost and the file in production were different documents.

It disallowed nine tokens: Amazonbot, Applebot-Extended, Bytespider, CCBot, ClaudeBot, CloudflareBrowserRenderingCrawler, Google-Extended, GPTBot, and meta-externalagent. Eight of those match the list Cloudflare documents for the feature.

The shape was accidentally correct. OAI-SearchBot was not on that list, and neither were Claude-SearchBot or PerplexityBot, so training was blocked and search was allowed, which is the outcome we would have chosen. Nobody here reasoned their way to it.

We have since turned it off, and the file our customers read is now the one in the repository. Two reasons, and the second matters more than the first. We are pre-launch with no archive to protect, so training inclusion is upside rather than cost. And an accidental right answer is still an unaudited one: a setting that changes production and appears in no diff will eventually change to something we would not have chosen, and nobody will notice.

If your site sits behind Cloudflare, the robots.txt your customers see may not be the one in your repository, and a code review will never catch the difference. Fetch the live file before you trust it.

How to check your own file in two minutes

Fetch it yourself first. curl -s https://yourdomain.com/robots.txt is the whole check, and it is the only version that counts, because your CDN, your host, or a security plugin may all be editing it after your code runs.

Then read for these three failures, in order of how much they cost:

1. A Disallow under OAI-SearchBot. This is the expensive one. You are opting out of ChatGPT citations for no training benefit.
2. A single group listing several AI bots by name. Common in copied snippets, and it means you cannot allow one without allowing all of them.
3. A token that no longer does anything. anthropic-ai is the usual one. Anthropic's current support article documents ClaudeBot, Claude-User, and Claude-SearchBot, and anthropic-ai is not among them. If it is your only Anthropic rule, ClaudeBot is not blocked and you probably think it is.

If you would rather not read the file by hand, our AI crawler checker fetches a live robots.txt and prints a verdict per token with the operator's own wording beside it. The robots.txt tester shows which group a given user agent actually matches. Both are free and need no account. Both are our own fetch and parse, not a paid API sitting behind an open endpoint.

What this does and does not tell you about visibility

Getting the tokens right makes you eligible. It does not make you cited. Those are different measurements, and conflating them is how people end up satisfied with a robots.txt audit and no idea whether they appear in an answer.

Worth saying plainly, because we sell a product in this space: SEOBuilder asks seven answer engines the same buyer questions, ChatGPT among them, and reports the citation URLs each answer showed. There is still no public API for what ChatGPT's search feature returned to a real user, so what we read is the product itself rather than a vendor endpoint. That is worth knowing about any number in this category, ours included: ask which prompts were run, and where the answer was read from.

For the measurement side of this, see how to measure AI visibility, which covers what is actually countable and what is not. If the vocabulary around this is still fuzzy, SEO vs AEO vs GEO sorts out which of those three labels describes real work.

Do this today: curl your production robots.txt, confirm OAI-SearchBot is not disallowed, and split any combined AI bot group into one group per token. That is a ten minute change, and for most sites it is the only AI crawler work worth doing this quarter.