How to add robots.txt, llms.txt and schema to a Webflow site

Before you start

You'll need three things:

1. Publish access to the Webflow site (any role above Reviewer can add schema, according to Webflow).
2. A paid Site plan or paid Workspace. Without one, the robots.txt field, custom code and the Schema markup field aren't available.
3. A custom domain connected, if you want llms.txt. Webflow doesn't publish it to the staging domain.

Have your three files ready before you open Webflow. If you don't have them yet, the free Web2Agents .txt generator builds a robots.txt and llms.txt you can paste straight in.

Step 1: Set your robots.txt rules

robots.txt tells crawlers, including AI search bots, which parts of your site they may access. One wrong line can hide your site from AI answers, so start here.

According to Webflow's guide to robots.txt rules:

1. Go to Site settings > SEO > Indexing.
2. Paste your rules into the robots.txt field.
3. Click Save, then publish your site.

A sensible starting point for a site that wants to be cited by AI assistants but opt out of AI training looks like this. Swap the private paths for your own, or delete them if you have none.

# AI search and user-triggered fetchers: allowed, so you can be cited
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /
Disallow: /checkout/

# AI training: opted out
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
Disallow: /

# Everyone else, including Googlebot and Bingbot
User-agent: *
Allow: /
Disallow: /checkout/

If you're happy for your content to be used in AI training, leave out the middle group. That's a business choice, not a technical one. Our own robots.txt at web2agents.com, for example, allows training bots too.

Three Webflow-specific details:

• Leave out the Sitemap: line. Webflow adds your sitemap to robots.txt by default. If you add it yourself, the file lists it twice. Webflow's sitemap guide explains how to turn the automatic line off if you really need a custom one.
• Repeat private paths in every named group. A crawler follows only the group that matches its name most specifically, so OAI-SearchBot won't read your User-agent: * rules. That's why /checkout/ appears twice above.
• Don't confuse this with the staging toggle. The Staging indexing switch in the same section only affects your webflow.io subdomain, according to Webflow's guide to disabling indexing.

Webflow's Indexing section can also send a Content-Signal header, such as ai-train=no, search=yes, to state how AI tools may use your content. Webflow notes this is based on a proposed extension to the robots.txt standard and is not yet an accepted standard. Treat it as an extra, not a replacement for robots.txt rules.

For the full reasoning behind which bots to allow, see our guide on allowing AI search bots while blocking AI training.

Step 2: Upload your llms.txt

llms.txt is a short Markdown file that gives AI agents a curated map of your key pages. Webflow added native support for it, announced in Webflow Updates.

According to Webflow's llms.txt help article:

1. Go to Site settings > SEO > LLMs.txt.
2. Click Upload file.
3. Choose your llms.txt file. It must be UTF-8 encoded and under 100 KB.
4. Click Save changes, then publish.

The file then loads at https://yourdomain.com/llms.txt. It won't appear on your webflow.io staging domain, so test it on the custom domain.

A minimal file for a Webflow site follows the structure in the llms.txt proposal. The studio and links below are made up:

# Northline Studio

> Northline Studio is a two-person design studio in Lisbon that builds Webflow websites for B2B software companies. Projects start at €4,000.

## Services

- [Webflow website design](https://www.northline.example/services/design): Full site design and build, 6–10 weeks
- [Webflow migrations](https://www.northline.example/services/migration): Moving WordPress sites to Webflow

## Work

- [Case studies](https://www.northline.example/work): Recent client projects with results

## Optional

- [About us](https://www.northline.example/about): Team and studio history

Webflow-specific limits:

• Webflow's help covers one llms.txt upload. The spec's optional .md versions of each page aren't supported. Webflow says a well-structured HTML page linked from llms.txt should be enough for now.
• There's no separate field for llms-full.txt in Webflow's help.
• Re-upload the file whenever you add or rename key pages. It doesn't update itself.

Keep expectations honest. Webflow's own help calls llms.txt "a new and emerging, experimental convention" and says it doesn't guarantee LLMs will read your site. An Ahrefs study of 137,210 domains found 97% of published llms.txt files received zero requests in May 2026. It takes five minutes, so it's worth doing, but only after robots.txt and your content are in good shape.

Step 3: Add schema markup (JSON-LD)

Schema markup tells search engines and AI tools what a page is about: who the business is, what a product costs, when an article was published. Webflow gives you two places to put it.

Site-wide schema: Organization and WebSite

Schema that describes your whole business belongs in the site head, so it's on every page. According to Webflow's guide to custom code in head and body tags:

1. Go to Site settings > Custom code.
2. Paste your script into the Head code section.
3. Click Save changes, then publish.

Here is a template. Replace every value with your own, and only list profiles that really belong to you in sameAs.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Organization",
      "@id": "https://www.yourdomain.com/#organization",
      "name": "Your Company",
      "url": "https://www.yourdomain.com/",
      "logo": "https://www.yourdomain.com/logo.png",
      "sameAs": [
        "https://www.linkedin.com/company/your-company"
      ]
    },
    {
      "@type": "WebSite",
      "@id": "https://www.yourdomain.com/#website",
      "name": "Your Company",
      "url": "https://www.yourdomain.com/",
      "publisher": { "@id": "https://www.yourdomain.com/#organization" }
    }
  ]
}
</script>

Google's Organization structured data guide lists the properties it reads, such as name, url, logo and sameAs.

Page-level schema, including CMS pages

For schema that belongs to one page, Webflow has a dedicated field. According to Webflow's schema markup guide:

1. Open the Pages panel and click the settings icon next to the page.
2. Scroll to Schema markup.
3. Paste your JSON-LD, or click Generate schema markup to let Webflow AI draft it.
4. Click Save, then publish.

On a Collection page (for example, your blog post template), bind CMS fields inside the JSON-LD so each item gets its own values, such as the post name as headline and the publish date as datePublished. Webflow notes that Reference, Multi-reference and Multi-image fields can't be used in schema. Supported fields include Plain text, Image, Link, Number, Date/Time and Option.

Two rules to avoid trouble:

• One place per schema. Webflow recommends against adding the same schema in both the Schema markup field and a custom code section, because it creates duplicate structured data.
• Check AI-generated schema. If you use Generate schema markup, read the result and make sure every value matches what's visible on the page.

What schema won't do: Google says you don't need special markup to appear in AI Overviews or AI Mode (AI features and your website), and Webflow notes that valid schema improves eligibility for rich results but doesn't guarantee them. Think of schema as accurate labelling, not a ranking trick.

Step 4: Publish and check everything

None of these settings go live until you publish. After publishing, check each one:

1. robots.txt: open https://yourdomain.com/robots.txt. You should see your rules plus one Sitemap: line added by Webflow.
2. llms.txt: open https://yourdomain.com/llms.txt on your custom domain, not the webflow.io one.
3. Schema: paste a page URL into the Schema Markup Validator to catch errors, and into Google's Rich Results Test to see which rich results the page qualifies for.
4. Search Console: if you changed robots.txt, Google says it generally caches the file for up to 24 hours, so give it a day before worrying.

Common Webflow mistakes

Get all three files ready for Webflow

Run the free Web2Agents scan to see whether your robots.txt blocks AI bots and how your homepage scores. To create the files yourself, use the free .txt generator. The AI-Ready Kit delivers a robots.txt, llms.txt, llms-full.txt and JSON-LD schema written for your site, with step-by-step install notes for Webflow.