Two kinds of AI traffic reach a site today. LLM crawlers read pages to train models or to answer a question someone asked a chatbot. AI agents go further: a model drives a real browser, clicks, types and submits forms on someone's behalf. They raise different questions, and they are best answered in different places.
LLM crawlers: decide at the edge
Crawlers read pages. Most of the large ones say who they are in the User-Agent and follow robots.txt, so that file is the first place to say what they may read. Your CDN can enforce it for the ones that ignore it, and some CDNs block or charge AI crawlers on their own. Content you sell, such as articles behind a paywall or a price list, needs more: the publishers' case is in bot protection for news sites.
Bot Protection does not keep a list of crawlers, and does not need to: it is asked about actions, not page views.
AI agents: decide at the action
An agent that books, signs up or buys does it through the same forms your customers use. Some of that is welcome, a person delegating a purchase to an assistant; some of it is the old abuse with a new driver, creating accounts, claiming free trials or trying passwords at volume. The page view tells you little. The sign-up, the login and the checkout are where the question matters, so that is where to ask:
- Sign-up and free trials: an agent that creates accounts in a loop is a sign-up bot, whatever drives it. Hold back the free credits for
bot; see free trial abuse. - Login: a second factor for
bot. An assistant acting for a real customer can pass it with them; a credential stuffer cannot. - Checkout: automation buying limited stock or testing cards is refused at
bot_high, whether a model or a script is at the wheel.
What gives a driven browser away
Most of today's agents drive Chrome through Playwright or the DevTools protocol, the same tools scrapers use. However human the model's choices, the browser still shows how it is driven:
- How the input arrives. A click with no mouse movement before it, keys released within a few milliseconds, text inserted without a key being pressed.
- The browser itself. Automation flags, a software GPU, a User-Agent that disagrees with the browser's own client hints.
- The pattern. Many requests on a few paths, fresh sessions in a row, one device behind many networks.
In our own tests in October 2026, fifteen automation tools, from plain scripts to Playwright with stealth flags and a real Chrome driven over the DevTools protocol, were all caught: most of their answers at a sign-in form were bot or bot_high.
Let the good ones through on purpose
If you want agents to buy from you, give them a door made for them: an API, or a step-up the person behind the agent can pass, such as a passkey. Bot Protection answers step_up when the evidence points both ways, and in monitor mode you can see how much of your traffic is automated before you block any of it.