Is llms.txt actually used?

15 October 2026 · By John Carlsson

Not by Google Search. On 15 June 2026 Google added a note to its documentation saying llms.txt files are not needed for Google Search and will neither help nor harm your rankings, because Google Search ignores them. Other tools may read the file at the moment a user asks something, but the crawler documentation from OpenAI, Perplexity and Anthropic does not describe llms.txt as a way to control crawling or to appear in their answers.

What llms.txt is

llms.txt is a proposal published by Jeremy Howard on llmstxt.org in September 2024. The site now describes a second version, updated after two years of adoption. It is not a web standard from the W3C or IETF.

The file is plain Markdown, placed at /llms.txt or at a sub-path such as /docs/llms.txt. The proposal sets out this structure:

An H1 with the name of the site or project. This is the only required part.
A blockquote with a short summary.
Optional paragraphs of detail.
H2 sections listing links, with a section called Optional for links an agent can skip.

The proposal also suggests offering clean Markdown copies of pages at the same URL with .md added. Its author says the file is mainly meant for use at inference time, when a tool fetches it to answer a question, rather than for training.

Google: not used by Google Search

Google's documentation updates log records that on 15 June 2026 it added a note to its AI optimisation guide clarifying how Google Search treats llms.txt files: they are not needed for Google Search.

The guide itself lists llms.txt under myths. It says you do not need new machine readable files, AI text files, markup or Markdown to appear in Google Search. It adds that it is fine to keep an llms.txt file for other services that use it, but that doing so will neither harm nor help your visibility or rankings in Google Search, because Google Search ignores these files.

Google's AI features page says the same about AI Overviews and AI Mode: no AI text files are required. Pages need to be indexed and eligible to show a snippet.

OpenAI, Perplexity and Anthropic

The official crawler documentation from the other major AI companies explains how to control their bots with robots.txt. None of the pages below describe llms.txt as a way to control their crawlers or to appear in their answers, although OpenAI and Perplexity do publish llms.txt files as indexes of their own documentation:

OpenAI's crawler overview covers OAI-SearchBot, GPTBot and ChatGPT-User. Its publishers FAQ says appearing in ChatGPT search depends on not blocking OAI-SearchBot.
Perplexity's crawler documentation covers PerplexityBot and Perplexity-User and recommends allowing PerplexityBot in robots.txt.
Anthropic's help article on its crawlers covers ClaudeBot, Claude-User and Claude-SearchBot and says its bots honour robots.txt.

That does not prove no tool ever reads llms.txt. It means none of these companies officially document it as a way to get into their answers, so there is no official basis for promising results from it.

Should you keep one?

It will not hurt you in Google, so the decision is about effort:

Keep it if you run documentation, an API or a software project where developers point AI coding tools at your site. That is the use the proposal was written for.
Keep it accurate. Every link should resolve, and it should not list pages you have removed or blocked.
Do not treat it as a substitute for robots.txt, crawlable HTML, internal links and clear content. Those are what Google and the AI crawlers document.
Do not pay for a service that sells llms.txt as a ranking fix for Google. Google has said in writing that it is ignored.

If you already have a file and want to check its structure and links against the proposal, the LLMs.txt Auditor on this site does that.

Sources

Frequently asked questions

Will an llms.txt file hurt my Google rankings?

No. Google's optimisation guide says it is completely fine to create and maintain llms.txt files for other services that use them, and that doing so will neither harm nor help your visibility or rankings in Google Search, because Google Search ignores them. The real cost is maintenance: if the file falls out of date it can point tools at old or missing pages.

Does robots.txt or llms.txt control AI crawlers?

robots.txt is the control. OpenAI, Perplexity and Anthropic all document user agents you can allow or block in robots.txt, such as OAI-SearchBot, GPTBot, PerplexityBot and ClaudeBot. llms.txt is a proposal for describing your content to language models. It does not grant or refuse access, and none of those crawler pages say it changes how they crawl.