CentriWebllms.txt is a proposed Markdown file, placed at yourdomain.com/llms.txt, that gives AI tools a short summary of a website and a list of links to its most useful pages. It is a proposal published by Jeremy Howard in September 2024, not an agreed web standard, and Google says Google Search does not use it, so for most business websites it is optional and comes after the basics.
It costs little to add and does no harm. What it does not do is get you into AI answers, block AI crawlers or replace pages that answer customers’ questions. This article explains what the file is, what goes in it, who is documented as reading it and how to decide whether your site needs one.
What is llms.txt?
The llms.txt proposal describes itself as a way to standardise on a file that helps AI agents use a website. The idea is simple. A large language model working on a question has limited room to read, and a web page full of menus, scripts and adverts wastes that room. A short Markdown file that says “this is who we are, and these are the pages that matter” is quicker for a tool to read than the whole site.
The proposal also suggests that sites offer a clean Markdown copy of each page at the same address with .md added, for example /services/hot-water.md, and describes how to point to those copies from the page.
The proposal’s own site calls it a proposal, describes its overview as informal and says the specification is open for community input. It is not run by a standards body. Treat it as a convention that some tools follow, not a rule every tool obeys.
What goes in an llms.txt file
The format is plain Markdown, in this order:
- A heading with the name of the business or site. This is the only part the proposal requires.
- A short summary in a quote block, a sentence or two saying what the site is about.
- Optional paragraphs or lists with more detail.
- Sections with second-level headings, each listing links in the form [page name](address), optionally followed by a colon and a note.
- An optional section headed “Optional” for links a tool can skip when it needs less.
The proposal says the file can sit at the site root or under a path, with a more specific file taking precedence.
An llms.txt example for a service business
For a plumbing business, a file might read like this:
# Example Plumbing
> Residential plumbing and hot water repairs for homes in the northern suburbs. Licensed plumbers, same-week bookings for most jobs.
Service area: the northern suburbs and nearby towns. Hours: weekdays 7am to 5pm, with an after-hours line for emergencies.
## Services
– [Hot water repairs and replacement](https://example.com/hot-water/): electric, gas and heat pump systems
– [Blocked drains](https://example.com/blocked-drains/): what we do, what affects the price
– [Leaking showers](https://example.com/leaking-showers/): repairs without retiling where possible
## About
– [Pricing and call-out fees](https://example.com/pricing/)
– [Contact and booking](https://example.com/contact/)
## Optional
– [Guides and articles](https://example.com/guides/)
Keep the summary factual and match it to your site. If the file says you work weekends and your contact page says you do not, you have given AI tools two answers.
Who actually reads llms.txt?
This is where most of the confusion sits, so separate what is documented from what is assumed.
Documented:
- Google Search does not use it. Google’s guide to optimising for generative AI features says Search ignores llms.txt and similar AI text files, and that creating one will neither help nor harm your visibility in Google Search. Its AI features page says you do not need AI text files to appear in AI Overviews or AI Mode.
- AI companies publish their own for developer documentation. OpenAI’s crawler page links to an llms.txt index of its documentation, and Perplexity’s crawler page points to one too. The llms.txt site says OpenAI, Anthropic and Google’s Gemini publish them for their developer docs.
- Coding and browsing agents. The proposal names coding agents that fetch documentation and chat assistants with search as uses, and says Chrome’s Lighthouse checks for the file as part of its agentic browsing audits.
Not documented: none of the crawler pages from OpenAI, Perplexity or Anthropic that we checked in November 2026 says their search products read llms.txt when choosing what to cite. Publishing your own docs as llms.txt is a different thing from reading everyone else’s for search answers. Anyone telling you the file improves your ChatGPT or Perplexity visibility is guessing.
llms.txt, robots.txt and sitemap.xml compared
The three files are often mixed up. They do different jobs.
| File | What it does | Who it is for | Can it block a crawler? |
|---|---|---|---|
| robots.txt | Tells crawlers which parts of the site they may visit | Search engines and AI crawlers | Yes, for crawlers that obey it |
| sitemap.xml | Lists the pages you want search engines to find | Search engines | No |
| llms.txt | Summarises the site and links its most useful pages | AI tools that choose to read it | No |
The proposal itself says robots.txt and llms.txt have different purposes, and that a sitemap is not a substitute because it often lists too much for a tool to read at once. If you want to stop an AI company training on your pages, that is a robots.txt decision. OpenAI’s GPTBot and Anthropic’s ClaudeBot are the training crawlers; see Anthropic’s crawler help article for its example rules.
Does your site need an llms.txt file?
For most small and mid-sized business websites, it is a nice-to-have. Weigh it this way.
Worth doing soon if:
- you publish software, an app or an API with documentation people read through AI coding tools
- your site has hundreds of pages and a clear list of the important ones would help
- the basics are already done and you want every reasonable signal in place
Can wait if:
- your service pages do not yet answer the main questions customers ask
- your robots.txt or security settings might be blocking AI search crawlers
- your business details differ between your site, your Google profile and directories
Those three items matter more for being found by AI tools, and our guides to getting recommended by ChatGPT and Google AI Overviews cover them. For the wider picture of what is known and what is guesswork, read our guide to AI SEO.
How to add llms.txt to a website
- Write it by hand first. Ten to thirty links is plenty for a service business. A file that lists every page defeats the point.
- Upload it to the root. On most hosting you can place the file in the main web folder through the hosting file manager or by asking your developer. On WordPress, some plugins can generate one; check the result rather than trusting the default.
- Check it loads. Visit yourdomain.com/llms.txt in a browser. It should show plain text.
- Keep it current. Add a reminder to review it whenever you add a service, change hours or retire a page. An out-of-date file is worse than none.
Add Markdown copies of pages only if your developer can produce them automatically. Maintaining two versions of every page by hand is not worth it for a small business.
What llms.txt cannot do
- It cannot guarantee that any AI tool reads it, cites you or recommends you.
- It cannot block crawling or training. Only robots.txt and similar controls do that, and only for crawlers that obey them.
- It cannot fix thin or unclear pages. A tool that follows your links still reads the pages themselves.
- It does not affect Google Search, by Google’s own statement.
- It is not private. Anyone can open the file in a browser, so leave out staff details, internal pages and anything you would not put on your home page.
- It does not replace structured data. If you want search engines to read your hours, address and services in a standard format, that is a job for schema markup, which Google does use for its normal search features.
CentriWeb sets up llms.txt as part of our AI search visibility and SEO work, after checking that crawlers can read the site and that the pages behind each link answer the questions customers ask.