What is llms.txt? A plain-English definition, with an example
llms.txt is a plain-text Markdown file, usually placed at yourdomain.com/llms.txt, that gives AI assistants and agents a short, curated map of a website's most important content. It was proposed in September 2024 by Jeremy Howard as a way to help language models find useful information without wading through navigation, ads and scripts. It doesn't block bots or control AI training. That is still the job of robots.txt.
- What: a Markdown file with your site's name, a one-line summary, and lists of links to key pages.
- Where: at the site root (
/llms.txt), or at a sub-path such as/docs/llms.txtcovering the pages under it. - Who proposed it: Jeremy Howard, published on llmstxt.org on 3 September 2024. A v2 was published in 2026.
- Status: a proposal, not an official web standard.
- What it isn't: an access rule, a training opt-out or a ranking signal.
- Does it get read? Rarely, so far. An Ahrefs study of 137,210 domains found 97% of published llms.txt files received zero requests in May 2026.
Where llms.txt came from
Web pages are built for people. An AI agent reading a page has to strip out menus, cookie banners and JavaScript to reach the actual information, and its context window can't hold a whole website. The llms.txt proposal suggests a simple fix: publish one small, well-structured Markdown file that points agents straight to the content that matters.
The idea borrows from files the web already uses. Just as /robots.txt and /sitemap.xml sit at a conventional location, /llms.txt gives agents a predictable place to look.
What goes in an llms.txt file
According to the specification, the sections come in this order:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a short summary of what the site is.
- Optional paragraphs or lists with extra context (no headings).
- H2 sections with lists of links, each written as
[name](url)followed optionally by: notes.
By convention, an H2 called Optional holds secondary links that an agent can skip when it needs a shorter context.
An example for a small business
Here is what an llms.txt could look like for a fictional bakery. The structure follows the spec. The business and links are made up.
# Rosa's Bakery
> Rosa's Bakery is an independent bakery in Porto, Portugal, selling sourdough bread, pastries and custom cakes. Open Tuesday to Sunday, with online pre-orders for pickup.
Prices are in euros and include VAT. Custom cake orders need 72 hours' notice.
## Main pages
- [Menu and prices](https://www.rosasbakery.example/menu): Full list of breads, pastries and cakes with current prices
- [Order online](https://www.rosasbakery.example/order): Pre-order for pickup, with available time slots
- [Custom cakes](https://www.rosasbakery.example/cakes): Sizes, flavours, allergens and how to order
## Policies
- [Opening hours and location](https://www.rosasbakery.example/visit): Address, hours and parking
- [Allergens](https://www.rosasbakery.example/allergens): Allergen information for every product
## Optional
- [Our story](https://www.rosasbakery.example/about): History of the bakery
What llms.txt is not
It's easy to expect too much from a file with "LLM" in its name. To be clear:
- It doesn't control access. Whether a bot is allowed to crawl your site is set in robots.txt. llms.txt has no allow or disallow rules.
- It doesn't opt you out of AI training. For that, you disallow training tokens such as GPTBot or Google-Extended in robots.txt.
- It isn't needed for Google's AI features. Google's guide to AI features says you don't need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode.
- It isn't a guaranteed ranking or citation boost. We haven't found any major AI search product stating that it uses llms.txt to choose what to cite.
Does anyone actually read llms.txt?
Here is the honest picture, from the best data we could find.
- Adoption is real, but uneven. In the Ahrefs study, 28% of the 137,210 domains it monitors publish an llms.txt file. Ahrefs notes its customers are more SEO-aware than average, so it treats that as an upper bound.
- Most files are never fetched. 97% of those files got zero requests in May 2026.
- When they are fetched, it's mostly by bots, and not always AI ones. Of the requests that did arrive, 19.5% came from named AI tools. GPTBot was the top AI fetcher, with Claude Code second. Ahrefs also points out that a fetch doesn't prove the file was used.
- Documentation is where it works best. The llmstxt.org proposal itself says llms.txt files are used most heavily for software documentation, where coding agents follow them to find API references. OpenAI, Anthropic and Google publish llms.txt files for their own developer docs.
The debate is lively among developers too. The original Hacker News discussion of llms.txt drew 175 comments and is a good place to see how developers weigh it up.
So, should you publish one?
Our view: yes, as a low-cost extra, not as a strategy.
It takes minutes to write, it's low-risk, and it gives agents a clean summary if they do look. Some platforms already do it for you: the llmstxt.org site lists Wix, which generates one for every site, and the Yoast SEO and AIOSEO plugins for WordPress.
But if you only have an hour to spend on AI visibility, spend it on the things that are proven to matter:
- A robots.txt that lets AI search bots in.
- Clear, well-structured pages that answer real questions.
- Accurate structured data and a working sitemap.
Then add llms.txt.
Create yours in a few minutes
You can build a robots.txt and llms.txt for your site with the free Web2Agents .txt generator, or read more in What is llms.txt? on our site. If you want files written for your specific pages, the AI-Ready Kit includes an llms.txt and llms-full.txt alongside robots.txt and schema.
FAQ
Is llms.txt an official standard?
No. It is a proposal published at llmstxt.org in 2024 and updated in 2026. It has not been adopted by a standards body.
Where do I put llms.txt?
At the root of your domain, so it loads at https://yourdomain.com/llms.txt. The spec also allows files at sub-paths, such as /docs/llms.txt.
What's the difference between llms.txt and llms-full.txt?
llms.txt is a short map with links. llms-full.txt is a companion convention that puts the full text of your key pages into a single file.
Does llms.txt replace robots.txt?
No. robots.txt says which bots may crawl which parts of your site. llms.txt only points agents to content. You need robots.txt either way.
Does Google use llms.txt?
Google's guidance for AI features in Search says no AI text files are needed to appear in AI Overviews or AI Mode.
Sources
- The /llms.txt file (proposal and specification), Jeremy Howard, llmstxt.org
- We analyzed 137K sites: 97% of llms.txt files never get read, Ahrefs
- AI features and your website, Google Search Central
- Llms.txt discussion, Hacker News
