OpenAI's crawler page is short and worth reading in full. The table below is what it says, in order of how much each bot matters to a business that wants to be cited.
| Bot | What it does | If you block it | robots.txt token |
|---|---|---|---|
| OAI-SearchBot | Indexes sites for ChatGPT search | Your site "will not be shown in ChatGPT search answers"; it may still appear as a plain link | OAI-SearchBot |
| GPTBot | Collects pages that may be used to train models | Signals that your content should not be used for training; search is unaffected | GPTBot |
| ChatGPT-User | Fetches a page when a user asks for it or a GPT action needs it | Nothing for search; OpenAI says robots.txt rules may not apply because a user asked | none documented |
| OAI-AdsBot | Checks landing pages submitted as ChatGPT ads | Only matters if you advertise | none documented |
The important sentence is this one: "OAI-SearchBot is for search." It builds the index that ChatGPT search answers from, and its robots.txt setting is independent of GPTBot's. Blocking GPTBot to keep your content out of training leaves you in search. Blocking OAI-SearchBot takes you out of search answers, and OpenAI says a change takes about 24 hours to show.
ChatGPT-User confuses people because it shows up in server logs. OpenAI says it "is not used for crawling the web in an automatic fashion" and "is not used to determine whether content may appear in Search". It is the fetch that happens when someone pastes your URL into ChatGPT or asks a question that needs your page right now.
OpenAI also notes that if a site allows both OAI-SearchBot and GPTBot, it may use one crawl for both purposes.
A robots.txt that keeps you in search
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /
The second block is optional. It keeps your pages out of training data without touching search. Many sites allow both; the choice is yours, and it has no effect on Google or Bing either way. Verify the bots by IP: OpenAI publishes ranges for each at openai.com/searchbot.json, gptbot.json and chatgpt-user.json.
