Should you block AI agents from your site? Cloudflare now lets you decide per agent
Not by default, and now you don’t have to treat them all the same. On 4 November 2026 Cloudflare added a Personal Assistants view that shows which AI agents visit your site for real people and what they request, plus a per-agent setting: Allow, Block on pages with ads, or Block. A customer’s agent comparing products or completing a purchase is still a customer, so the new controls let you welcome the useful ones.
- Cloudflare’s Security Analytics now shows Personal Assistants as its own traffic category, next to Verified Bot, Likely Human, Likely Automated and Automated.
- A new Personal Assistants widget lists which agents visit and how many requests each sends, with the paths, status codes and networks for each one.
- Site owners can set an overall Agents policy and a different one for each supported personal agent: Allow (do not block), Block on pages with ads, or Block.
- The setting is renamed AI Crawler and Agent access policies. Existing Search and Training policies stay as they are.
What happened
In a blog post on 4 November, Cloudflare described personal agents as the go-between that turns a person’s request into actions on websites: comparing products, finding support articles or completing a purchase. To a site owner, that has all looked like anonymous automated traffic. Cloudflare’s point is that “blocking agents altogether can feel like the safest choice, but it can mean losing customers.” Its changelog confirms the per-agent controls for the top personal agents.
What this means for your store or business
- Look before you block: if your site is on Cloudflare, check which agents already visit and whether they reach your product pages and checkout or hit errors.
- Decide per agent: the options let you allow agents that bring buyers, and use “Block on pages with ads” if ads pay for your content.
- Fix the pages agents fail on: errors on pages an agent can’t use mean lost requests from real customers. Our guide on checking which AI crawlers can access your site covers the other places bots get blocked.
The technical details
The policies live under Security > Settings > AI Crawler and Agent access policies, and new Cloudflare sites are asked during setup whether they make money from ads, with recommended policies to match. Sites using Account Abuse Protection, which is in early access, can also see which user account each agent acts for and limit one account without blocking the agent for everyone. Cloudflare says its next step is carrying user context, such as whether the person is an existing customer, through the agent.
Want to know whether AI agents can find and use your site? Run our free AI check.
FAQ
Sources
- See the agents, set the terms: new visibility and control for personal agents, Cloudflare blog
- Granular access controls for the top Personal Agents, Cloudflare changelog
