llms.txt
llms.txt is a plain-text file placed at the root of a website that points AI crawlers and assistants to the pages and information you most want them to use.
It works a little like a sitemap, but for large language models: a simple, curated map of your most important content. Unlike robots.txt, it controls no access. It is a proposed convention, no engine has confirmed that it affects citations, and it may help only the tools that choose to read it.
How it works
The file lives at /llms.txt and is written in Markdown: a title, a one-paragraph summary of the site, then sections of links, each with a short description of what the page covers. A tool that supports it can read it in one request and understand what the site is about and where the authoritative pages are, without crawling everything.
It is a proposed convention rather than an official standard, so support varies by engine, but it costs little to publish and keep current.
Example
troiana.net's llms.txt is generated from the filesystem on every request: capabilities, insights, tools, glossary terms, the AI Hub and company pages are listed automatically, so a newly published article appears in it without anyone editing the file.
In practice
Check whether you already have one by visiting your domain followed by /llms.txt. If nothing loads, write a short version: the business name, one paragraph saying what you do and for whom, then a list of perhaps ten to thirty key pages, such as services, pricing, about, contact and your best guides, each with a one-line description. Keep the descriptions factual and identical in substance to what the pages say. It should read like a helpful briefing note for a new colleague.
Decide who updates it. A file written once and forgotten will soon point to pages you have removed or renamed, which helps nobody. If your developer can generate it automatically from the site's content, that removes the maintenance problem, but check the output so it lists what matters rather than every tag page. Remember that llms.txt does not control access: crawlers that you want to keep out must be blocked in robots.txt, and anything you put in llms.txt is public. Do not use it for private notes, internal links or instructions aimed at manipulating answers.
Why it matters for your business
llms.txt is cheap insurance rather than a growth lever. Writing one takes an hour or so for a small site, it does no harm, and it forces a useful exercise: deciding which pages genuinely represent your business. The honest limit is that public evidence of AI engines relying on it is thin, and support varies by engine, so it will not rescue a site that engines cannot crawl or do not trust. Your robots.txt rules, a correct XML sitemap and clear, crawlable pages matter far more. Publish one if it is easy, keep it current, and do not pay a premium for it.
Common mistakes
- Listing every URL on the site instead of the pages that matter.
- Letting the file go stale so it points to removed or redirected pages.
- Expecting it to override robots.txt; it guides crawlers, it does not grant access.
Questions people ask
Does Google use llms.txt?
Google has not said that it uses llms.txt for Search or AI Overviews, and its public guidance on appearing in AI features points to ordinary SEO practice instead. Some AI tools and developer assistants do read the file. Treat it as a low-cost extra for engines that choose to use it, not a Google requirement.
How is llms.txt different from an XML sitemap?
An XML sitemap lists every page you want crawled, in a machine format, so search engines can discover them. llms.txt is a short, curated, human-readable summary written in Markdown that explains what the site is and links only to the most important pages with descriptions. One aims at completeness, the other at understanding.
Can I add llms.txt to a WordPress site?
Yes. Upload a plain-text file named llms.txt to the root folder of your site, so it loads at yourdomain/llms.txt, or use a plugin that generates one from your pages. Generated versions are easier to keep current, but check that they list your important pages rather than every post and tag archive.