OAI-SearchBot

OAI-SearchBot is OpenAI's separate crawler that indexes web content specifically for ChatGPT Search, building the retrieval pool that feature cites from, independent of the GPTBot crawler used for model-training data collection. Blocking OAI-SearchBot removes a page from ChatGPT Search's citation pool without affecting training opt-outs.

Why it matters

For any brand actively pursuing ChatGPT Search visibility, confirming OAI-SearchBot has unblocked access is a more direct and consequential check than confirming GPTBot access, since OAI-SearchBot is the crawler that actually feeds citations rather than training data.

OpenAI's crawler documentation states that sites which opt out of OAI-SearchBot will not be shown in ChatGPT search answers, although they can still appear as navigational links, and that robots.txt changes take about 24 hours to apply. ChatGPT-User, which fetches pages for user requests, is not used to decide what appears in search.

Its activity is growing fast: in a Botify analysis of seven billion log events, OAI-SearchBot requests rose 3.5 times in the eight months after GPT-5 launched in August 2025. Rate-limiting or blocking it to save server load can cost ChatGPT citations, as our article on AI crawler rate limits explains.

Related terms

• GPTBot

  GPTBot is OpenAI's crawler for collecting web content used to train its models, operating separately from the crawlers OpenAI uses to power ChatGPT Search citations. A site can block GPTBot specifically to opt out of having its content used for model training, without affecting whether ChatGPT Search can still retrieve and cite that same content.

• ChatGPT Search

  ChatGPT Search is OpenAI's web-grounded answer mode inside ChatGPT, which retrieves current web pages before generating a response and displays a source panel alongside the answer. It launched as a standalone feature in late 2024 and uses a separate crawler, OAI-SearchBot, to build its citation index apart from GPTBot's model-training crawl.

• ChatGPT-User

  ChatGPT-User is the user-agent OpenAI applies when ChatGPT fetches a specific page live, in real time, at a user's explicit request within a conversation, distinct from the background indexing done by GPTBot or OAI-SearchBot. It represents an on-demand fetch rather than a systematic crawl.

• robots.txt (AI directives)

  robots.txt is the long-standing standard crawler-control file at a site's root, now extended with AI-specific user-agent rules that separately allow or disallow crawlers like GPTBot, ClaudeBot, and PerplexityBot. A single robots.txt file can permit standard search crawling while blocking one or more AI crawlers, or the reverse.