AI bots
Google-Agent: what it is, its IP ranges, and how to block it
Google-Agent is Google's crawler for AI search and answers.
| Vendor | |
|---|---|
| What it is for | AI search and answers |
| In its User-Agent | Google-Agent |
| Its vendor's address list | https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json |
| IP ranges | 20 |
It reads a page for an AI's search or for a person who asked an assistant, and can bring visitors. Bot Protection lets it through by default; you can block it.
How to verify it
Any bot can write Google-Agent in its User-Agent. A request is Google-Agent only when its address is in Google's published list below: check the address, not the name.
How to block it
With robots.txt, you ask it to stay out:
User-agent: Google-Agent Disallow: /
robots.txt is a request, not a block, and it cannot tell the real crawler from a bot wearing its name. Bot Protection verifies Google-Agent by its address on every request and answers it as you choose on the Set-up page, per kind or for Google-Agent alone; a bot that only wears the name is judged like any other bot.
Its IP ranges
From Google's own list, downloaded again every day.
IPv4
136.121.16.0/24 136.121.24.0/21 136.121.40.0/21 136.122.0.0/16 74.125.232.0/28 74.125.232.112/28 74.125.232.16/28 74.125.232.32/28 74.125.232.48/28 74.125.232.64/28 74.125.232.80/28 74.125.232.96/28
IPv6
2001:4860:c::/124 2001:4860:c::10/124 2001:4860:c::20/124 2001:4860:c::30/124 2001:4860:c::40/124 2001:4860:c::50/124 2001:4860:c::60/124 2001:4860:c::70/124