Insights · Field note · Growth and AI visibility
llms.txt explained: what it is, what it is not
A root text file that describes the site to language models. Cheap, harmless where ignored, and widely misunderstood.
llms.txt is a proposed convention: a Markdown file at a site’s root that describes the organization and lists its most important pages, optionally with a companion llms-full.txt listing every page with a one-line description. It gives language model systems a curated map. It is not a ranking signal, not a substitute for crawlable pages, not a way to control training use, and not uniformly read by every engine. Its value is modest and real, and it is only worth having if it is generated from the build so it never drifts.
In detail
What the evidence shows.
What the proposal says
The llmstxt.org proposal specifies a Markdown file at /llms.txt with a title, a short description, and sections of links with one-line descriptions, written for language models rather than for browsers. A fuller index, /llms-full.txt, can list every page. The format is deliberately simple so that it can be generated and read without tooling.
What it is not
It does not make a page citable; the page does that. It does not block or permit crawling or training; robots.txt and crawler-specific tokens do that. It is not read by every engine, and adoption changes over time. Claims that it transforms visibility should be treated with the same skepticism as any other single-file fix.
Why publish it anyway
It costs little, it is harmless where ignored, and some retrieval systems do read it. For a large site it offers a curated entry point and a complete index that are more useful than a sitemap for a system choosing what to fetch. This site publishes both files.
How to do it properly
Generate both files from the content source on every build. Exclude drafts, noindex pages and redirects. Validate every link. Serve as plain text. Review the descriptions quarterly. The specification checklist covers each step; see also llms.txt and machine-readable indexes.
Sources
Primary references.
By David Selakovic · Published 2026-10-16 · Revised 2026-10-16 · Reviewed every 180 days