llms.txt is a plain Markdown file at the root of a website, such as example.com/llms.txt, that summarizes the site and links to its most useful pages so that AI systems can find the right content quickly. Jeremy Howard of Answer.AI proposed it in September 2024. It is a proposal, not a standard, and as of 2026-10-20 none of OpenAI, Anthropic, Google or Perplexity has said that its assistants read it.
Should you add one? If your platform generates it for you, yes, because it costs nothing and cannot hurt. If it means custom work, do the things that measurably matter first. This guide covers the format, who reads the file today, what the assistants themselves say about it, and how to write one in ten minutes.
We make Seen42, an AI visibility tool, so read our product notes with that in mind. Everything else in this guide works without it.
What the file looks like
The proposal at llmstxt.org defines a small structure:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a one or two sentence summary.
- Any number of H2 sections, each a list of Markdown links with an optional note after a colon.
- An H2 called "Optional" for secondary links an agent can skip when it needs less context.
A complete file for a small software company fits on one screen:
# Acme Payroll
> Acme Payroll runs payroll for UK agencies with 5 to 50 staff. Monthly plans, no setup fee.
## Product
- [Pricing](https://acme.example/pricing): Plans, limits and what each includes.
- [Integrations](https://acme.example/integrations): Xero, QuickBooks and FreeAgent.
- [Acme vs Competitor](https://acme.example/compare/acme-vs-competitor): Differences, with sources.
## Optional
- [Changelog](https://acme.example/changelog): Release notes by month.
Keep descriptions factual, serve the file as plain text with a 200 status, and update it when the pages it points to change. A longer companion file, llms-full.txt, is sometimes used to publish the full text of documentation in one place; it is useful for developer tools and rarely for anyone else.
Who reads it today
Documentation platforms and developer tools adopted the file first, because an agent reading API docs benefits from a curated index. Several site builders and SEO plugins now generate it automatically. That is the extent of confirmed use.
On the assistant side, Google's John Mueller has said that no AI system he knew of used llms.txt, comparing it to the keywords meta tag: a claim by the site owner that the system can check by reading the site itself. Google's own documentation on AI features says you do not need "new machine readable files, AI text files, or markup" to appear in AI Overviews or AI Mode. OpenAI, Anthropic and Perplexity document their crawlers in detail and do not mention llms.txt.
We asked all five assistants "What is llms.txt and should I add it to my website?" on 2026-10-20. Every one of them described it as a proposed, optional convention. ChatGPT cited the GitHub repository of the proposal; Perplexity cited Search Engine Land, Semrush and the documentation of site builders that generate the file. None claimed that it affects whether a site appears in answers. Perplexity's answer put it plainly: add it if it is easy to maintain, and do not expect it to improve rankings or guarantee inclusion.
So why have one
Three reasons, in order of weight.
It is a courtesy to the agents and developer tools that do read it. That is a small group today, concentrated around documentation sites, and a curated list saves them work.
It forces you to decide which pages matter. Writing twelve links with one-line descriptions is a useful exercise even if nobody reads the result, because it is the same list you should be linking from your navigation and footer.
It is nearly free when generated. If your site builder or publishing tool produces it from the same data as your sitemap, there is no reason not to publish it.
What not to expect
Do not expect it to replace robots.txt, which is what the crawlers actually obey, or a sitemap, which is what search engines actually read. Do not expect it to change what ChatGPT or Perplexity say about you this quarter. Do not put anything in it that is not on the site; the file is most useful when it is a faithful index, and an agent that finds a mismatch has learned not to trust it.
How to add one
- List the ten to twenty pages that answer buyer questions: pricing, product, integrations, comparisons, the methodology or trust page.
- Write a one-line factual description of each.
- Save the file as Markdown at the root of the domain, publicly readable, served as text.
- Regenerate it whenever those pages change, ideally from the same data that builds your sitemap.
- Check it in a browser once a month.
Where Seen42 fits
Seen42 checks the prompts your buyers ask on ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews, shows who is named and cited instead of you, and builds and hosts comparison, facts and guide pages from your own facts. Paid plans start at $39 a month for 15 prompts; the Free plan checks 10 prompts once on ChatGPT and Gemini. Every site Seen42 hosts publishes a sitemap, robots.txt and an llms.txt generated from the same page data, so the file is always current without anyone maintaining it. We publish one for seen42.com the same way.
