How to check your website’s AI visibility for free in 2026
Start with the free Web2Agents AI check. It scores your homepage out of 30, checks robots.txt, sitemap.xml and llms.txt, and tests 12 AI bots against your robots.txt rules. Then use the manual checklist below to confirm that your content is in the raw HTML, that a firewall is not blocking the bots you allowed, and that ChatGPT, Perplexity and Gemini actually mention your brand.

- Two things to check: whether AI crawlers can reach and understand your pages, and whether AI answers mention your brand. The free check covers the first. Mentions you confirm yourself, by asking the assistants.
- The free Web2Agents AI check is the place to start: enter your homepage URL and an email, and you get a score out of 30 across six areas, on screen in real time, plus a PDF. Nothing to install, no account and no card. It only reads your public homepage and public files such as robots.txt, sitemap.xml and llms.txt.
- It reads robots.txt rules, not live firewall access: the check tests 12 AI bots, including GPTBot, ClaudeBot and PerplexityBot, against your robots.txt. A firewall or anti-bot service can still block a bot that the file allows.
- It does not query ChatGPT, Perplexity or Gemini to see whether they cite you. That step is in the DIY checklist.
- Search bots and training bots are different switches: OpenAI treats OAI-SearchBot (ChatGPT search) and GPTBot (training) as independent. Blocking a search crawler can hide you from answers; blocking a training crawler is a business choice.
What a good AI visibility check must cover
AI visibility is not one score. A useful check answers five practical questions:
| What to check | Why it matters | What the free Web2Agents check does | What you should also do by hand |
|---|---|---|---|
| AI crawler access in robots.txt | If a search crawler is disallowed, that assistant may never index you. OpenAI says sites opted out of OAI-SearchBot “will not be shown in ChatGPT search answers”, except as navigational links | Tests 12 AI bots against your robots.txt rules | Read the file yourself and separate search bots from training bots |
| Sitemap and llms.txt | A sitemap helps crawlers find your pages. llms.txt is a short Markdown map that can help agents use a site; it is not an access or training control | Checks whether sitemap.xml and llms.txt are present | Keep the sitemap honest, and treat llms.txt as a useful extra, not a ranking switch |
| Structured data | Schema markup labels facts such as your organisation, products and policies so machines can read them consistently | Structured data is one of the six scoring areas | Keep markup in line with the text on the page |
| Content in the raw HTML | Many AI crawlers read the HTML they are served, without running the JavaScript a browser runs | The check scores homepage content and structure; it does not claim to compare a browser view with a no-JavaScript view | View the page source and search for a sentence from the middle of the page |
| Mentions in AI answers | Access is not the same as being cited. You also need to see what ChatGPT, Perplexity and Gemini actually say | It does not query those assistants | Ask them your customers’ questions and note the sources they cite |
| Live server access | robots.txt is a request. A CDN, firewall or host can still return a 403 or a challenge page | It does not send live requests as each crawler; the report notes that firewalls can still block agents | Test access and read your logs, or use the AI-Ready Kit’s 19-bot live table |
Start with the free check so you have a structured reading of the homepage. Then close the gaps the check does not cover.
Start with the free Web2Agents AI check
- Open the free AI check.
- Enter your homepage URL and an email address, and agree to occasional emails. That consent is a condition of the check.
- Wait for the results on screen. You can also download a PDF. The check does not email the report.
The score is out of 30, across six areas:
| Area | What it is looking at, in plain terms |
|---|---|
| Content semantics | Whether the page has a clear structure and meaning: headings, language and descriptive text a machine can follow |
| Structured data | Whether the page describes itself in a machine-readable format, such as JSON-LD |
| Navigability | Whether a crawler can find and fetch the page, including robots.txt and sitemap signals |
| Actionability | Whether a visitor or an agent could take a next step, such as contacting you or completing a form |
| AI promptability | Whether the content is written and delivered in a way an assistant can quote |
| Agent collaboration | Whether the site makes its AI policy explicit, including files such as llms.txt and rules for AI bots |
The check only looks at your public homepage and the public files next to it. It will not log in, buy anything or scan your whole site.
Two honest limits, so you do not over-read the score:
- Homepage only. A strong homepage does not prove that product, location or policy pages are equally clear. For up to 25 pages and 36 checks, use the €19 AI-Ready Kit or work through our free GEO audit checklist.
- Rules, not arrival. The 12-bot test reads robots.txt. It does not prove that each bot received a 200 response from your server. If a search bot you wanted to allow is marked as blocked in the file, fix that first. If the file allows the bot, still confirm live access with the checklist below.
A manual DIY checklist
Do these after the free check, or on their own if you want a second pass.
1. Read your robots.txt
Open https://yourdomain.com/robots.txt and look for the bots that decide whether you can be found and cited:
| Allow if you want to be found in AI answers | Your choice (training or use of crawled content) |
|---|---|
OAI-SearchBot (ChatGPT search) |
GPTBot (OpenAI training) |
Claude-SearchBot and Claude-User |
ClaudeBot (Anthropic training) |
PerplexityBot |
Google-Extended (Gemini training and grounding in Gemini Apps; it does not affect Google Search) |
A crawler follows only the most specific User-agent group that matches it. If you add a named group for OAI-SearchBot, that bot no longer reads your User-agent: * rules, so repeat any private-path Disallow lines in the named group.
For the full list and a copy-paste template, see how to allow AI search bots but block AI training and our AI crawler list.
2. Confirm the text is in the page source
Open a key page, choose View Source (not the browser inspector), and search for a sentence from the middle of the page: a price, a policy line or a product spec. If it is missing, the text may only appear after JavaScript runs. Google asks for important content “in textual form” for its AI features in Search. Other AI crawlers may also miss content that exists only after scripts run.
Do the same for reviews, prices and shipping terms if they sit in a widget or a pop-up.
3. Ask the assistants your customers’ questions
This is the mention check the free tool does not run.
Write 10 questions a real customer would ask, in their words, with a place, a use or a constraint if that is how they search. Ask them in ChatGPT, Perplexity and Gemini (and Google AI Mode, if you care about Google’s AI features). Note:
- whether you are named, and how you are described
- which of your pages, if any, are cited
- any wrong hours, prices, domains or policies
Answers change from one run to the next, so look for patterns, not a single result. Repeat the same list every month. Our checklist for getting cited by ChatGPT covers what to do if you are missing.
4. Check your logs for real bot visits
If you have access or CDN logs, search for OAI-SearchBot, GPTBot, ClaudeBot, Claude-SearchBot and PerplexityBot. Look at the status codes. A run of 403s or challenge pages means something in front of the site is turning the bot away, even if robots.txt allows it.
User-agent names can be faked. OpenAI publishes IP ranges for its bots, so you can tell a real GPTBot or OAI-SearchBot visit from an impostor. Anthropic also publishes a source-IP list and warns that blocking those addresses may stop its bots from reading your robots.txt, which is a poor way to opt out.
No visits does not always mean you are blocked. A new or small site may not have been crawled yet.
5. Test live access if the logs look wrong
A robots.txt Allow is permission, not arrival. To test what your server returns, send a request with each bot’s user agent and compare the body with a normal browser response. A 200 that contains a “verify you are human” page is still a block.
This test from your own computer only shows how the site treats a user-agent name from your IP. Some firewalls also check the visitor’s address, so confirm with logs and official IP lists. Our guide on checking which AI crawlers can access your website walks through the steps, and we have a separate note on firewalls blocking AI bots that robots.txt allows.
The €19 AI-Ready Kit includes a 19-bot access table that checks both robots.txt and what the server returned.
What to fix first
If you only have an afternoon, work in this order:
- Let the AI search bots in. If robots.txt or your firewall blocks
OAI-SearchBot,PerplexityBot,Claude-SearchBotor Googlebot, fix that before anything else. You can still opt out of training at the same time. - Put the important facts in the HTML. Prices, hours, policies and product details should be readable in the page source.
- Give crawlers a sitemap, and add an llms.txt if it is cheap to do. Google says you do not need new AI text files or special markup to appear in AI Overviews or AI Mode. llms.txt can still help agents that look for it. Our explainer on what llms.txt is sets the expectations, and robots.txt vs llms.txt vs sitemap.xml shows how the three files differ.
- Add or correct structured data so it matches the visible text.
- Ask the assistants again after the technical fixes have had time to be recrawled.
If you need a robots.txt or llms.txt written for you, use the free robots.txt and llms.txt generator. Review and edit the files before you save them.
Check your site
Run the free Web2Agents AI check for a homepage score out of 30, on screen. When you want the fixes applied to more pages, the €19 AI-Ready Kit (a launch price, usually €29, including VAT) runs 36 checks on up to 25 pages, ranks the work, and includes ready-to-upload robots.txt, llms.txt and schema files. More guides are in our GEO and AI search visibility and AI crawlers and robots.txt sections.
FAQ
What is AI visibility, and how do I check it for free?
AI visibility is whether AI search tools and assistants can reach your pages, understand them and mention your brand in answers. Start with the free Web2Agents AI check for a homepage score and a robots.txt reading, then ask ChatGPT, Perplexity and Gemini your customers’ questions and check your logs.
What does the free Web2Agents AI check actually test?
It scores your public homepage out of 30 across six areas: content semantics, structured data, navigability, actionability, AI promptability and agent collaboration. It also checks robots.txt, sitemap.xml and llms.txt, and tests 12 AI bots against your robots.txt rules. Results appear on screen in real time, with a PDF you can download.
Does the free check show whether ChatGPT mentions my brand?
No. It does not query ChatGPT, Perplexity, Gemini or other assistants. To check mentions, ask those tools the questions your customers ask and note whether you are named and which sources they cite.
Can a firewall block AI bots that robots.txt allows?
Yes. robots.txt only asks well-behaved bots to stay away. Your CDN, firewall or host decides what is actually served. If they block a bot, it can get a 403 or a challenge page even when robots.txt says Allow: /.
Will a good score get my site cited by ChatGPT?
No check can guarantee a citation. A good score means the obvious technical barriers are lower. Whether an assistant cites you also depends on your content, your search rankings and what other sites say about you.
Do I need an account to run the free check?
No account and no card. You do need to enter an email address and agree to occasional emails. The check only reads your public homepage and public files.
Sources
- Overview of OpenAI crawlers, OpenAI Developers
- Does Anthropic crawl data from the web, and how can site owners block the crawler?, Claude Help Center
- Google’s common crawlers, Google Crawling Infrastructure (includes Google-Extended)
- AI features and your website, Google Search Central
- The /llms.txt file, llmstxt.org
- Web2Agents free AI check, Web2Agents
- AI-Ready Kit: what’s included, Web2Agents


















