llms.txt explained
What is llms.txt?
llms.txt is a Markdown file at the root of a website that tells large language models what the site is about and which pages to reference.
llms.txt meaning
`llms.txt` is a plain-text Markdown file placed at the root of a domain, for example `https://example.com/llms.txt`. It gives large language models a clean, structured summary of a site: what it is, the most important pages, and short descriptions of each. Think of it as a `robots.txt` for AI, but instead of controlling crawling, it presents your best content in a format that models can read easily.
The standard was proposed in September 2024 and is written in Markdown on purpose. Markdown is compact, token-efficient, and easy for both humans and models to parse, which makes it a natural fit for feeding context to AI systems.
Why llms.txt matters for AI search
AI assistants and answer engines like ChatGPT, Perplexity, Claude, and Google AI Overviews increasingly summarize and cite web pages. A clear llms.txt helps these tools understand your site structure, reference the right URLs, and describe your content accurately. This is part of a wider shift toward GEO (generative engine optimization) and AEO (answer engine optimization), where the goal is to be quoted correctly by AI, not only ranked in classic search.
How an llms.txt file is structured
The format is simple Markdown: an H1 with the site name, a short summary, then H2 sections with links and one-line descriptions. A minimal example looks like this:
# Example Project
Short description of what the site or product does.
## Docs
- [Getting started](https://example.com/start): install and first steps
- [API reference](https://example.com/api): endpoints and examples
## Optional
- [Changelog](https://example.com/changelog): release historySome sites also publish an `llms-full.txt` that includes the full text of key pages for models that want deeper context.
llms.txt vs robots.txt vs sitemap.xml
| File | Audience | Purpose |
|---|---|---|
| robots.txt | Search crawlers | What they may or may not crawl |
| sitemap.xml | Search engines | A full list of URLs to index |
| llms.txt | AI models | A curated, readable summary of key content |
They are complementary. robots.txt and sitemap.xml help traditional SEO, while llms.txt is aimed at AI consumption. Markdown Docs publishes its own llms.txt as a working example.
How to create an llms.txt file
- List your most valuable pages: docs, product, key guides.
- Write a one-line description for each, in plain language.
- Format the file in Markdown with an H1 title and H2 sections.
- Save it as `llms.txt` and upload it to your site root.
- Keep it current as you add important pages.
Because it is just Markdown, you can draft and preview it in the online Markdown editor before publishing.
Is llms.txt an official standard?
llms.txt is a proposed community standard, not an official requirement. As of 2026, the major AI providers have not formally confirmed they read it during crawling. Even so, it is low-cost to publish, it documents your site clearly, and it positions you for a future where AI systems increasingly rely on structured, citable sources. Many documentation platforms now generate it automatically.
Related Markdown files
llms.txt is part of a growing family of Markdown files that give machines context. See common .md files for AI projects, or learn about AGENTS.md and CLAUDE.md for instructing AI coding agents.
Related references
- llms.txt proposal (llmstxt.org) for the original specification.
- Semrush: What is llms.txt for adoption context and SEO view.
FAQ
Where should llms.txt be placed?
Put llms.txt at the root of your domain, such as https://yoursite.com/llms.txt, so AI tools can find it in a predictable location.
Is llms.txt the same as robots.txt?
No. robots.txt controls what search crawlers may access. llms.txt presents a curated, Markdown summary of your best content for large language models.
Do I need llms.txt for SEO?
It is optional and not yet officially used by major AI providers, but it is cheap to publish, documents your site clearly, and supports AEO and GEO strategies.
What format does llms.txt use?
It uses Markdown: an H1 site name, a short summary, then H2 sections containing links with one-line descriptions.