What is GPTBot?
GPTBot is the web crawler OpenAI uses to collect content for training its models. OpenAI’s crawler documentation, read on 13 November 2026, says GPTBot “is used to make our generative AI foundation models more useful and safe” and “to crawl content that may be used in training our generative AI foundation models.” Disallowing it in robots.txt “indicates a site’s content should not be used in training generative AI foundation models.”
GPTBot is not the bot that decides whether ChatGPT search can cite your page. That is OAI-SearchBot. Several of the pages that rank for “gptbot” describe GPTBot as the crawler that powers ChatGPT’s answers, or list OpenAI agents that OpenAI’s page no longer mentions and leave out OAI-SearchBot. OpenAI’s own page treats them as separate switches.
How does GPTBot differ from OpenAI’s other bots?
OpenAI lists four user agents on the same page. Each has a different job and its own IP list.
| User agent | What OpenAI says it does | Respects robots.txt | Used for training |
|---|---|---|---|
| GPTBot | Crawls content “that may be used in training our generative AI foundation models” | Yes | Yes, unless you disallow it |
| OAI-SearchBot | Surfaces websites “in search results in ChatGPT’s search features” | Yes | Not described as a training crawler |
| ChatGPT-User | Visits a page when a user asks ChatGPT or a Custom GPT a question; “not used for crawling the web in an automatic fashion" | "Robots.txt rules may not apply,” because a user started the action | Not described as a training crawler |
| OAI-AdsBot | Visits pages submitted as ads on ChatGPT to check them against OpenAI’s policies | Only visits submitted ad pages | No. OpenAI says its data “is not used to train generative AI foundation models” |
OpenAI adds two details that matter for site owners. First, if a site allows both GPTBot and OAI-SearchBot, OpenAI “may use the results from just one crawl for both use cases to avoid duplicative crawling.” Second, robots.txt changes for search take about 24 hours to reach OpenAI’s systems.
What does GPTBot look like in your logs?
OpenAI gives this example user-agent string, and notes that the version number may change:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
When GPTBot fetches your robots.txt file, OpenAI says it may add a robots.txt marker to the string so you can tell those requests apart in logs that do not record paths.
GPTBot IP ranges
OpenAI publishes GPTBot’s IP addresses at openai.com/gptbot.json. On 13 November 2026 the file listed 18 IPv4 ranges, with a creation time of 22 September 2026. The other bots have their own files: searchbot.json, chatgpt-user.json and adsbot.json.
Anyone can put “GPTBot” in a user-agent string. To confirm a request is really OpenAI’s, check that the IP falls inside the published ranges. If your firewall or CDN blocks bots by default, allow these ranges explicitly for the agents you want.
How do you allow or block GPTBot?
GPTBot reads its own group in robots.txt. This file blocks training while keeping the site open to ChatGPT search:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
A crawler follows only the group that names it, so if you also have private paths in a User-agent: * group, repeat them in each named group. Our free robots.txt generator writes these groups for OpenAI and the other AI companies and tests any URL against the result.
Should you block GPTBot?
The decision is about training, not traffic.
| Your situation | Typical policy |
|---|---|
| B2B marketing site that wants buyers to hear accurate facts about the product | Allow GPTBot and OAI-SearchBot |
| Content is the product (paid research, data, a publisher) | Disallow GPTBot, allow OAI-SearchBot |
| Private areas (app, admin, staging) | Disallow every crawler on those paths |
Blocking GPTBot does not remove you from ChatGPT search, and allowing it does not get you cited there. Which pages ChatGPT search cites, and how, is covered in our ChatGPT SEO guide. For how Perplexity’s crawler compares, see Perplexity SEO, and for the wider practice of being cited in AI answers, see GEO.
Common questions about GPTBot
What is GPTBot?
OpenAI's web crawler for model training. OpenAI's crawler documentation says GPTBot 'is used to crawl content that may be used in training our generative AI foundation models' and that disallowing it 'indicates a site's content should not be used in training generative AI foundation models.'
Does blocking GPTBot remove my site from ChatGPT?
No. OpenAI says each of its robots.txt settings is independent: a site can allow OAI-SearchBot to appear in ChatGPT search results while disallowing GPTBot. Sites that block OAI-SearchBot, not GPTBot, are the ones OpenAI says will not be shown in ChatGPT search answers, though they can still appear as navigational links.
Does GPTBot follow robots.txt?
Yes. OpenAI documents GPTBot as a robots.txt token webmasters use to manage training access. The exception among OpenAI's agents is ChatGPT-User, which visits a page when a user asks; OpenAI says that because these actions are user-initiated, robots.txt rules may not apply.
How do I verify that a visit is really GPTBot?
Check both the user-agent string and the IP address. OpenAI publishes GPTBot's IP ranges at openai.com/gptbot.json; on 13 November 2026 the file listed 18 IPv4 ranges and was dated 22 September 2026. A request that says GPTBot but comes from an IP outside those ranges is not OpenAI's crawler.
What are OpenAI's crawlers and user agents?
OpenAI's documentation lists four: OAI-SearchBot (surfaces sites in ChatGPT search), GPTBot (training), ChatGPT-User (fetches pages for user actions in ChatGPT and Custom GPTs) and OAI-AdsBot (checks landing pages submitted as ChatGPT ads). Each has its own published IP list.
Should a B2B company block GPTBot?
It is a choice about training, not about traffic. Blocking GPTBot keeps your future pages out of OpenAI's training data but does not change whether ChatGPT search can cite you. Companies whose content is the product, such as paid research, often block training and allow search; most B2B marketing sites allow both so models learn accurate facts about their product.



















