llms.txt is one of the most talked-about files in AI search, and one of the most misunderstood. Some vendors sell it as a way to get into ChatGPT. The evidence does not support that.
This guide explains what the file is, what the spec says, who reads it, and whether it is worth your time.
What is llms.txt?
llms.txt is a proposed standard for a Markdown file, placed at /llms.txt, that gives large language models and AI agents a short, curated overview of a website with links to clean, LLM-friendly versions of its key pages. It helps agents navigate a site; it is not a ranking or citation signal.
The idea was proposed by Jeremy Howard in September 2024 and is documented at llmstxt.org. A revised v2 of the proposal was published in August 2026.
The problem it targets is simple. Web pages wrap information in navigation, scripts and ads. Converting that HTML into clean text is imprecise, and context windows are limited. llms.txt offers a compact map instead, written in Markdown that both people and machines can read.
It is often compared to robots.txt and sitemap.xml, but it does a different job. robots.txt says what bots may access. A sitemap lists every indexable URL. llms.txt is a curated reading list for an agent working on a task.
What does the llms.txt specification require?
The spec requires only one thing: an H1 with the name of the site or project. Everything else is optional, but must appear in a set order if used.
According to llmstxt.org, a compliant file contains, in order:
- An optional byte-order mark.
- An H1 with the project or site name (the only required section).
- A blockquote with a short summary of the site.
- Zero or more Markdown sections, such as paragraphs or lists, with more detail. No headings are allowed here.
- Zero or more H2-delimited sections containing "file lists": Markdown links in the form
name, optionally followed by a colon and a note.
A section titled "Optional" is a convention for secondary links an agent can skip when it needs less context. In v2 it no longer carries special mechanical meaning.
The file can live at the site root or at a subpath such as /docs/llms.txt. A file covers the pages under its path, and the most specific file applies.
What does an llms.txt file look like?
A good llms.txt file is short: a title, a one-line summary, a few notes, and grouped links to your most useful pages. Here is an illustrative example for a service business:
# Example Agency
> Example Agency is a digital marketing agency offering SEO, answer engine optimization and web development for small and mid-sized businesses.
Key facts: founded in 2015, headquartered in Austin, Texas. All pricing is custom-quoted.
## Services
- [SEO services](https://example.com/services/seo.md): Technical, on-page and content SEO
- [Answer engine optimization](https://example.com/services/aeo.md): Visibility in ChatGPT, Perplexity and Google AI features
## Guides
- [What is AEO?](https://example.com/blog/what-is-aeo.md): Definitions, tactics and measurement
## Optional
- [About us](https://example.com/about.md)
- [Contact](https://example.com/contact.md)Notice that links point to .md versions of pages. The spec recommends linking to LLM-friendly content rather than full HTML pages. The FastHTML documentation file cited on llmstxt.org is a real-world reference.
What is llms-full.txt?
llms-full.txt is a companion convention that puts the full text of a site's documentation into one Markdown file, rather than just linking to it. It is not part of the llmstxt.org spec itself.
The pattern spread through documentation platforms. Mintlify, for example, generates both files: llms.txt as an index, and llms-full.txt that "combines your entire documentation site into a single file as context for AI tools".
The two serve different needs:
| File | Contains | Best for |
|---|---|---|
| llms.txt | Curated index with links and notes | Agents that fetch only what they need |
| llms-full.txt | Full text of all included pages | Pasting a whole doc set into a model's context |
| Page .md twin | Clean Markdown of one page | Agents following a link from llms.txt |
For most business websites, llms-full.txt adds little. It matters mainly for software documentation, where developers load whole doc sets into coding assistants.
What are Markdown twins and why do they matter?
A Markdown twin is a clean .md copy of a web page, served at the same URL with .md added, so agents can read the content without parsing HTML. The llms.txt spec treats these as a core part of the proposal.
The spec allows two URL forms: .md appended to the full page URL (page.html.md) or the extension replaced (page.md). URLs without a file name use index.html.md or index.md.
Version 2 added discovery hints. Per llmstxt.org, a page can point to its Markdown version with rel="alternate" type="text/markdown", and to the covering llms.txt file with rel="describedby". Both can be sent as an HTML link element or an HTTP Link header.
Markdown twins are arguably more useful than llms.txt itself. They give any agent following a link a clean version of the page. Meek Media's blog posts, for example, have a Markdown twin at the same URL with .md added.
Keep twins identical in substance to the HTML page. Serving different content to machines and people risks being treated as cloaking.
Who actually reads llms.txt today?
Coding agents and developer tools use llms.txt most; there is no public evidence that AI search engines use it to choose citations. The spec itself says the files are used most heavily for software documentation, where coding agents follow them.
On the adoption side, the evidence is real:
- OpenAI, Anthropic and Google publish llms.txt files for developer docs: OpenAI, Anthropic and Gemini API.
- Chrome Lighthouse includes an llms.txt audit in its agentic browsing checks. It flags server errors when fetching the file; a missing file is rated not applicable.
- WordPress plugins such as Yoast and AIOSEO can generate the file, as listed on llmstxt.org.
On the search side, the evidence is thin. Google's John Mueller said in April 2025 that no AI services had said they use llms.txt, comparing it to the keywords meta tag. Publishing developer docs in the format is not the same as using it for search ranking.
Does llms.txt improve AI citations or SEO?
No reliable evidence shows that llms.txt increases citations in ChatGPT, Perplexity or Google's AI features. Treat it as optional infrastructure, not a visibility tactic.
Google's documentation is explicit: "You don't need to create new machine readable files, AI text files, or markup" to appear in AI Overviews or AI Mode.
The largest public study agrees. SE Ranking analyzed nearly 300,000 domains and found about 10% had an llms.txt file. It found no link between having the file and how often a domain was cited by LLMs; its model performed better with the variable removed. The study noted GPTBot sometimes fetches the files, though not often.
What drives citations instead is covered in how ChatGPT chooses which brands to recommend and What is answer engine optimization?: crawler access, clear answers, evidence and third-party mentions.
So why bother? Because it is cheap, harmless and useful to the agents that do read it.
How do you create an llms.txt file?
Creating llms.txt takes under an hour for most sites: list your key pages, write a summary, format it to the spec, and upload it to your root directory. Follow these steps.
- Pick 10–30 key pages. Services, products, pricing policy, top guides, about and contact. Skip thin or outdated pages.
- Write the H1 and summary. Use your exact brand name and a one-sentence description matching your other profiles.
- Add key facts as plain paragraphs or lists: location, founding year, what you do and do not offer.
- Group links under H2 headings with a short note for each.
- Link to Markdown twins if you have them; otherwise link to HTML pages.
- Upload to /llms.txt and serve it as plain text with a 200 status.
- Test it by asking an agent about your business with only the file as a starting point, as the spec suggests.
- Keep it current by regenerating it from your CMS or build step.
Review auto-generated output. Tools often dump every URL, which defeats the point of a curated list.
Should you add llms.txt to your website?
Add llms.txt if it is cheap for you to maintain, but do not expect it to change your AI search visibility. The decision depends mostly on who your audience is.
| Situation | Recommendation |
|---|---|
| Software product with developer docs | Yes — docs are the spec's main use case |
| Content-heavy site with a CMS plugin | Yes, if generation is automatic |
| Small service business | Optional; fix basics first |
| Site with crawl or rendering problems | Not yet — fix access first |
The priority order for most businesses is clear. First, make sure search and AI crawlers can reach you and that content is in the raw HTML. Second, restructure key pages so each section answers a real question. Third, build reviews and third-party mentions. llms.txt and Markdown twins come after that, as a finishing layer.
This mirrors the wider shift described in AI SEO: what actually changes: fundamentals still carry the weight, and new files are supporting infrastructure.
Frequently asked questions
Does Google use llms.txt?
Google has not said it uses llms.txt for Search, and its documentation states no AI text files are needed to appear in AI Overviews or AI Mode. Google does publish an llms.txt for its Gemini API docs, and Lighthouse checks the file for errors, but neither means it influences rankings or AI citations in Search.
Does ChatGPT read llms.txt?
OpenAI has not documented llms.txt as an input to ChatGPT search. ChatGPT search relies on OAI-SearchBot and third-party search providers. SE Ranking reported GPTBot occasionally fetches llms.txt files, but found no link between having one and being cited. ChatGPT can still read the file if a user or agent points it there.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which URLs they may access and is widely honored by search bots. llms.txt grants or blocks nothing; it is a curated Markdown guide to your content for AI agents. Keep managing crawler access through robots.txt, and use llms.txt only as an optional navigation aid.
Can llms.txt hurt my SEO?
A correctly formatted llms.txt file should not hurt SEO. It is a plain text file that search engines do not treat as a ranking signal. Problems only arise if linked Markdown pages contradict your HTML content, create thin duplicates, or return errors. Keep content consistent and URLs working.
Should a small business website have an llms.txt?
It is optional. If your site is small, your time is better spent on crawler access, clear service pages, reviews and a complete Google Business Profile. If your CMS or SEO plugin can generate llms.txt in a few clicks, there is little downside to adding one.
How often should I update llms.txt?
Update it whenever you add, remove or rename key pages, or when core facts change, such as services or locations. Generating it automatically from your CMS or build process is the easiest way to keep it accurate. A stale file that links to missing pages is worse than none.
Wondering whether files like llms.txt are the real issue, or whether something else is keeping AI engines from citing you? Meek Media's AI visibility audit checks crawler access, rendering, Markdown readiness and how AI engines describe your brand, starting with a free audit. Our technical SEO and web development teams can then implement fixes, or you can ask us a question.