SEO & ContentUpdated Nov 13, 2026

GPTBot

GPTBot is OpenAI's web crawler for model training. OpenAI says it crawls content that may be used to train its generative AI foundation models, and that disallowing GPTBot in robots.txt tells OpenAI a site's content should not be used for that training. It is separate from OAI-SearchBot, the crawler behind ChatGPT search, and from ChatGPT-User, which fetches pages when a user asks.

Related search gptbotwhat is gptbotgptbot user agent

What is GPTBot?

GPTBot is the web crawler OpenAI uses to collect content for training its models. OpenAI’s crawler documentation, read on 13 November 2026, says GPTBot “is used to make our generative AI foundation models more useful and safe” and “to crawl content that may be used in training our generative AI foundation models.” Disallowing it in robots.txt “indicates a site’s content should not be used in training generative AI foundation models.”

GPTBot is not the bot that decides whether ChatGPT search can cite your page. That is OAI-SearchBot. Several of the pages that rank for “gptbot” describe GPTBot as the crawler that powers ChatGPT’s answers, or list OpenAI agents that OpenAI’s page no longer mentions and leave out OAI-SearchBot. OpenAI’s own page treats them as separate switches.

How does GPTBot differ from OpenAI’s other bots?

OpenAI lists four user agents on the same page. Each has a different job and its own IP list.

User agentWhat OpenAI says it doesRespects robots.txtUsed for training
GPTBotCrawls content “that may be used in training our generative AI foundation models”YesYes, unless you disallow it
OAI-SearchBotSurfaces websites “in search results in ChatGPT’s search features”YesNot described as a training crawler
ChatGPT-UserVisits a page when a user asks ChatGPT or a Custom GPT a question; “not used for crawling the web in an automatic fashion""Robots.txt rules may not apply,” because a user started the actionNot described as a training crawler
OAI-AdsBotVisits pages submitted as ads on ChatGPT to check them against OpenAI’s policiesOnly visits submitted ad pagesNo. OpenAI says its data “is not used to train generative AI foundation models”

OpenAI adds two details that matter for site owners. First, if a site allows both GPTBot and OAI-SearchBot, OpenAI “may use the results from just one crawl for both use cases to avoid duplicative crawling.” Second, robots.txt changes for search take about 24 hours to reach OpenAI’s systems.

What does GPTBot look like in your logs?

OpenAI gives this example user-agent string, and notes that the version number may change:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

When GPTBot fetches your robots.txt file, OpenAI says it may add a robots.txt marker to the string so you can tell those requests apart in logs that do not record paths.

GPTBot IP ranges

OpenAI publishes GPTBot’s IP addresses at openai.com/gptbot.json. On 13 November 2026 the file listed 18 IPv4 ranges, with a creation time of 22 September 2026. The other bots have their own files: searchbot.json, chatgpt-user.json and adsbot.json.

Anyone can put “GPTBot” in a user-agent string. To confirm a request is really OpenAI’s, check that the IP falls inside the published ranges. If your firewall or CDN blocks bots by default, allow these ranges explicitly for the agents you want.

How do you allow or block GPTBot?

GPTBot reads its own group in robots.txt. This file blocks training while keeping the site open to ChatGPT search:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

A crawler follows only the group that names it, so if you also have private paths in a User-agent: * group, repeat them in each named group. Our free robots.txt generator writes these groups for OpenAI and the other AI companies and tests any URL against the result.

Should you block GPTBot?

The decision is about training, not traffic.

Your situationTypical policy
B2B marketing site that wants buyers to hear accurate facts about the productAllow GPTBot and OAI-SearchBot
Content is the product (paid research, data, a publisher)Disallow GPTBot, allow OAI-SearchBot
Private areas (app, admin, staging)Disallow every crawler on those paths

Blocking GPTBot does not remove you from ChatGPT search, and allowing it does not get you cited there. Which pages ChatGPT search cites, and how, is covered in our ChatGPT SEO guide. For how Perplexity’s crawler compares, see Perplexity SEO, and for the wider practice of being cited in AI answers, see GEO.

Common questions about GPTBot

What is GPTBot?

OpenAI's web crawler for model training. OpenAI's crawler documentation says GPTBot 'is used to crawl content that may be used in training our generative AI foundation models' and that disallowing it 'indicates a site's content should not be used in training generative AI foundation models.'

Does blocking GPTBot remove my site from ChatGPT?

No. OpenAI says each of its robots.txt settings is independent: a site can allow OAI-SearchBot to appear in ChatGPT search results while disallowing GPTBot. Sites that block OAI-SearchBot, not GPTBot, are the ones OpenAI says will not be shown in ChatGPT search answers, though they can still appear as navigational links.

Does GPTBot follow robots.txt?

Yes. OpenAI documents GPTBot as a robots.txt token webmasters use to manage training access. The exception among OpenAI's agents is ChatGPT-User, which visits a page when a user asks; OpenAI says that because these actions are user-initiated, robots.txt rules may not apply.

How do I verify that a visit is really GPTBot?

Check both the user-agent string and the IP address. OpenAI publishes GPTBot's IP ranges at openai.com/gptbot.json; on 13 November 2026 the file listed 18 IPv4 ranges and was dated 22 September 2026. A request that says GPTBot but comes from an IP outside those ranges is not OpenAI's crawler.

What are OpenAI's crawlers and user agents?

OpenAI's documentation lists four: OAI-SearchBot (surfaces sites in ChatGPT search), GPTBot (training), ChatGPT-User (fetches pages for user actions in ChatGPT and Custom GPTs) and OAI-AdsBot (checks landing pages submitted as ChatGPT ads). Each has its own published IP list.

Should a B2B company block GPTBot?

It is a choice about training, not about traffic. Blocking GPTBot keeps your future pages out of OpenAI's training data but does not change whether ChatGPT search can cite you. Companies whose content is the product, such as paid research, often block training and allow search; most B2B marketing sites allow both so models learn accurate facts about their product.

Start with one process.

We learn the business and build your first go-to-market process on us this month.