AI bots
The AI crawlers, and how to tell the real ones
Each AI crawler below names itself in its User-Agent, and its vendor publishes the addresses it crawls from. Anyone can write the name; only the address proves it. The ranges here are the vendors' own lists, downloaded again every day, the same ones Bot Protection verifies every request against.
| AI bot | Vendor | What it is for | IP ranges |
|---|---|---|---|
GPTBot | OpenAI | AI training | 18 |
OAI-SearchBot | OpenAI | AI search and answers | 39 |
ChatGPT-User | OpenAI | AI search and answers | 234 |
OAI-AdsBot | OpenAI | AI search and answers | 2 |
ClaudeBot | Anthropic | AI training | 14 |
Claude-User | Anthropic | AI search and answers | 14 |
Claude-SearchBot | Anthropic | AI search and answers | 14 |
PerplexityBot | Perplexity | AI search and answers | 8 |
Perplexity-User | Perplexity | AI search and answers | 4 |
CCBot | Common Crawl | AI training | 5 |
Google-CloudVertexBot | AI search and answers | 317 | |
Google-Agent | AI search and answers | 20 | |
DuckAssistBot | DuckDuckGo | AI search and answers | 486 |
MistralAI-User | Mistral AI | AI search and answers | 4 |
MistralAI-Index | Mistral AI | AI search and answers | 2 |
AI training crawlers take pages to train models and send nobody back; AI search and answers crawlers read a page for an AI's search or for a person who asked an assistant. Crawlers whose vendors publish no addresses are not listed: their name cannot be verified.