Learn · AI crawlers
What is GPTBot?
By Cloute · Published October 1, 2026
The short answer
GPTBot is OpenAI's web crawler for collecting content that may be used to train its generative AI models. Blocking it in robots.txt tells OpenAI not to use a site for training, but it does not remove the site from ChatGPT search, which uses a separate crawler, OAI-SearchBot.
GPTBot visits in Cloute's server logs of local business pages in 2026
busiest AI crawler in that dataset
visits from OpenAI's three crawlers combined
What does GPTBot do?
OpenAI says GPTBot "is used to crawl content that may be used in training our generative AI foundation models" and that disallowing it "indicates a site's content should not be used in training generative AI foundation models." It is a training crawler. It does not decide what appears in ChatGPT's search answers.
What is GPTBot's user agent?
OpenAI publishes this user agent string:
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
- robots.txt token: GPTBot
- IP ranges: published by OpenAI at openai.com/gptbot.json, so a request claiming to be GPTBot can be checked.
How is GPTBot different from OAI-SearchBot and ChatGPT-User?
| Crawler | What OpenAI says it is for | Follows robots.txt |
|---|---|---|
| GPTBot | Content that may be used to train OpenAI's models | Yes |
| OAI-SearchBot | Surfacing websites in ChatGPT's search results | Yes |
| ChatGPT-User | Visiting a page when a ChatGPT user's request calls for it | May not apply, because a user started it |
OpenAI also says that when a site allows both GPTBot and OAI-SearchBot, it may use the results of one crawl for both purposes to avoid crawling twice.
How do I block GPTBot?
Add these lines to the robots.txt file at the root of your domain:
- User-agent: GPTBot
- Disallow: /
This affects training only. To stay in ChatGPT's search answers, leave OAI-SearchBot allowed.
Should a local business block GPTBot?
It is a preference, not a visibility decision. Blocking GPTBot keeps future content out of OpenAI's training data. It does not stop ChatGPT from finding, opening or recommending the business when someone asks, because that runs through OAI-SearchBot and ChatGPT-User.
Like the other AI crawlers, it does not run JavaScript. A page whose text only appears after scripts load looks empty to it, so the content should be in the HTML the server sends. Cloute found its own website was invisible to AI crawlers for exactly this reason in 2026, and fixed it.
Read next
Research: 80,000+ AI crawler visits to local business websites What is OAI-SearchBot? What is ChatGPT-User? AI crawlers: the complete list for 2026 Should a local business block AI crawlers?Questions people ask
Does blocking GPTBot remove my site from ChatGPT?
No. OpenAI uses OAI-SearchBot for ChatGPT search results and ChatGPT-User for pages opened during a conversation. Blocking GPTBot only signals that your content should not be used for training.
Is GPTBot visiting local business websites?
Yes. GPTBot made 13,991 visits in Cloute's server logs of local business pages in 2026, the second most of any AI crawler.
How can I tell a real GPTBot from a fake one?
Check the requesting IP address against the list OpenAI publishes at openai.com/gptbot.json. Anyone can copy the user agent string.
Does GPTBot run JavaScript?
No. Content that only appears after JavaScript runs is not seen by AI crawlers, so put it in the HTML the server sends.
About Cloute. Cloute is an AI visibility company for local businesses. It measures how often ChatGPT, Gemini, Claude, Perplexity and Google AI Overviews recommend a business against its local competitors, and works every month to move it to the top.