llms.txt on WordPress: what it is, who reads it and whether your site needs one
llms.txt is a proposed markdown file at /llms.txt that lists a site's main pages for AI agents. It is not a standard. Google says Search ignores it, and no AI company's crawler page says its crawler reads yours. An SEO plugin may already have made one, so check first.
- By
- WP Ministry
- Published
- Tested on
- WordPress 7.1.3, PHP 8.3.35
In short
- llms.txt is a proposal on llmstxt.org, published in September 2024 and revised in August 2026. It is a markdown list of a site's main pages, and it is not a standard.
- Google's documentation says Search ignores the file, so it neither helps nor harms there.
- The crawler pages of OpenAI, Anthropic and Perplexity, and Bing's Webmaster Guidelines, do not say their crawlers read it. Publishing one for their own documentation is a different thing.
- All in One SEO makes one by default. Yoast SEO, Rank Math and SEOPress PRO make one once the feature is switched on.
- WordPress itself makes none. With Plain permalinks the address can still answer 200 with the home page, so read the content type and not only the status.
- The file grants and blocks nothing. Access is decided by robots.txt and by whatever stands in front of the site.
llms.txt is a proposed text file, written in markdown and placed at /llms.txt, that lists a site's most important pages for AI agents. It is a proposal published on llmstxt.org and not a standard, and WordPress does not make one. An SEO plugin may. All in One SEO makes one by default, and Yoast SEO, Rank Math and SEOPress PRO make one once the feature is switched on.
Whether anything reads the file is a separate question, and this page answers it from each company's own documentation. Google says Search ignores it. The crawler pages of OpenAI, Anthropic and Perplexity, and Bing's Webmaster Guidelines, do not say their crawlers read one. The file costs little, and nothing an engine has published says it changes whether your pages are quoted. So find out whether your site already has one before anyone sells you one.
What llms.txt is
The proposal lives at llmstxt.org. That page names Jeremy Howard as its author, and gives September 3, 2024 as the day it was published and August 10, 2026 as the day it was last modified. The current text is version 2, which the proposal's page of changes dates to August 2026. It describes itself in one line: "A proposal to standardise on using an /llms.txt file to provide information to help agents use a website."
The file is ordinary markdown in a fixed order:
| Part | Required | What it holds |
|---|---|---|
| A first-level heading | Yes | The name of the site or project |
A quoted paragraph (>) | No | A short summary of what the site is |
| Paragraphs or lists | No | Anything else a reader needs to make sense of the links |
| Sections under second-level headings | No | Lists of links, each a markdown link with an optional note after a colon |
A section named "Optional" is, by the proposal's convention, for links that can be skipped. The file sits at the root of the site, or in a folder, where it covers the pages under that folder.
The idea, as the proposal puts it, is that an agent reads the short file first and then follows only the links it needs. The proposal says such files are used most heavily for software documentation, where coding agents follow them to find references and tutorials. Version 2 also proposes a markdown copy of each page at the same address with .md added, and two link relations that point to those copies and to the file.
It is a proposal, not a standard
The proposal's own page calls it a proposal throughout. It closes by saying the specification is open for community input, and that a GitHub repository hosts what it calls an informal overview. It names no standards body. The plugins that generate the file say the same in their own documentation. Rank Math's reads: "The llms.txt is currently a proposal and not an official standard." The changelog of the SEOPress release that added the feature carries a disclaimer that the file is not yet officially supported by major search engines or AI platforms.
What it is not
| robots.txt | A sitemap | llms.txt | |
|---|---|---|---|
| What it is for | Telling crawlers which addresses they may request | Listing a site's pages for search engines | Pointing an agent to a short list of pages worth reading |
| Does it allow or block anything | Yes, for crawlers that honor it | No | No |
| Where it is defined | RFC 9309, a standards-track document of the IETF | The Sitemap protocol at sitemaps.org | One author's proposal at llmstxt.org |
The proposal draws the same lines itself. It says robots.txt tells automated tools what access is acceptable, while llms.txt is read on demand when an agent needs information, and that a sitemap lists all of a site's indexable pages, which is too much to stand in for a short list. Nothing you write in llms.txt keeps a crawler out or lets one in. Where robots.txt is in WordPress and the sitemap at wp-sitemap.xml cover the other two files.
Who reads it, according to each company
Each row is what that company's own page said when it was read on October 8, 2026. A page that does not mention the file is reported as exactly that. Silence is not a statement that a crawler ignores the file, and it is not support either.
| Company | Its page | What it says about llms.txt |
|---|---|---|
| The guide to generative AI features in Search, last updated July 10, 2026 | Search does not use such files, including for its generative AI features | |
| OpenAI | "Overview of OpenAI Crawlers", undated | Nothing about reading a site's file. It describes four user agents and robots.txt |
| Anthropic | Its help article on crawling, dated April 7, 2026 | Nothing. It describes three bots and robots.txt |
| Perplexity | "Perplexity Crawlers", undated | Nothing about reading a site's file. It describes two user agents and robots.txt |
| Microsoft | Bing Webmaster Guidelines, undated, which cover Bing, Copilot and grounding results | Nothing. Its guidance of October 8, 2025 on AI search answers does not mention the file either |
Google is the only one of the five that states a position. Its guide lists llms.txt among things a site owner can ignore for Search, and the documentation updates log records a note added on June 15, 2026 to make that plain. The guide says it is fine to keep such a file for other services that use one, and then: "Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." It adds that Google may crawl and index many kinds of files, and that finding the file does not mean it is treated in a special way.
Two other parts of Google do touch the file, and neither is Search. Chrome's Lighthouse has an llms.txt audit, whose documentation, last updated May 5, 2026, calls the file an emerging convention, marks a site with no file as not applicable because providing one is optional, and flags only a server error at that address. And the Gemini API's documentation publishes an llms.txt of its own.
OpenAI, Anthropic and Perplexity publish the file for their own documentation, which is not the same as their crawlers reading yours. On OpenAI's and Perplexity's crawler pages, the only mention of llms.txt is a pointer to the index of that company's own developer documentation. Anthropic publishes one for its developer documentation and one for its help center. Each of those files is an index of the company's own pages. None of the crawler pages says that OAI-SearchBot, GPTBot, ClaudeBot, Claude-SearchBot or PerplexityBot fetches /llms.txt from the sites it visits, or does anything with one.
That an assistant "supports" llms.txt is said widely. On the pages above it is documented by nobody.
What was studied
One study with data is worth knowing, with its limits.
SE Ranking, a company that sells SEO software including tools that track AI search, published it on November 7, 2025. It reports on nearly 300,000 domains. Of those, 10.13% had the file. Its authors found no correlation between having the file and how often a domain was cited in the AI answers they sampled, and say that taking the file out of their prediction model made the model more accurate.
The article does not name the assistants sampled or the dates of the sample, and its own disclaimer says the findings depend on the model and dataset tested. It is a finding of no relation in one dataset, not proof that the file can never matter. It has not been reproduced here.
How a WordPress site comes to have one
WordPress does not make an llms.txt. On a fresh WordPress with no plugin active and any permalink structure other than Plain, the address answers with the site's "Page not found" page and a 404. The check further down covers Plain permalinks.
Four SEO plugins can make one. Each line below is from that plugin's own documentation or source.
| Plugin | Since | On by default | What it makes |
|---|---|---|---|
| Yoast SEO | 25.3, released June 10, 2025 | No. Its help page has you switch it on | A real file in the site's root, refreshed weekly. In the free plugin. Not available on multisite |
| Rank Math | Not stated in its documentation | No. A module you switch on, and not among the modules a new install starts with | No file. WordPress answers the address on request. Empty until you choose what goes in it |
| All in One SEO | Not stated in its documentation | Yes | llms.txt, and on the same screen a switch for llms-full.txt and an option to convert posts to markdown |
| SEOPress | 9.5, announced January 27, 2026 | No. You activate it | Part of SEOPress PRO |
So a site that runs All in One SEO has the file unless someone switched it off, and a site that runs one of the others has it if someone switched the feature on. Installing or updating an SEO plugin can therefore be the whole of what an "llms.txt setup" amounts to.
What goes into a generated file is the plugin's choice. Yoast's specification says it includes the five latest updated posts, pages and custom post types, cornerstone content first, and that pages can be picked by hand instead. Rank Math's documentation says you choose the post types and it lists up to 100 items by default, leaving out posts set to noindex.
Check whether your site has one
Step 1: Ask for the address
Replace
example.comwith your own address. This prints the status and the content type of the answer and nothing else.bashcurl -sL -o /dev/null -w '%{http_code} %{content_type}\n' https://example.com/llms.txtStep 2: Read what it prints
What it prints What it means 404 text/html; charset=UTF-8There is no llms.txt. This is what a fresh WordPress answers 200 text/plain, with or without a charset after itSomething is serving an llms.txt: a file or a plugin 200 text/html; charset=UTF-8Not an llms.txt. A page of the site answered at that address The third line is the one that misleads. On a WordPress with Plain permalinks, on a server that still hands such requests to WordPress,
/llms.txtis redirected to/llms.txt/and answered with the home page and a 200. A tool that looks only at the status reports a file that is not there. Any permalink structure other than Plain brings the 404 back.Step 3: Read the file
An llms.txt starts with a line that begins
#and reads as a list of links. Check that every address in it is one you want listed and that none of them is a draft, a private page or a page that no longer exists.bashcurl -sL https://example.com/llms.txtStep 4: Find out what made it
Look in the site's root, the folder that holds
wp-admin,wp-contentandwp-includes, over SFTP or in the host's file manager. A file namedllms.txtthere was put there by a person or written there by a plugin, as Yoast SEO does. If the address answers and there is no file, a plugin is answering through WordPress, as Rank Math does. The plugin's settings, described below, say which.
Make one without a plugin
A static file is enough. Write it in a plain text editor, save it as llms.txt in lowercase, and upload it to the site's root.
# Example Plumbing Co.
> Example Plumbing Co. repairs and installs plumbing for homes and small businesses. This file lists the pages that say what we do and how to reach us.
Opening hours and the area we cover are on the contact page.
## Services
- [Emergency repairs](https://example.com/services/emergency-repairs/): What counts as an emergency and how to call us out
- [Water heater installation](https://example.com/services/water-heater-installation/): What an installation includes
## Company
- [About us](https://example.com/about/): Who we are and the area we cover
- [Contact](https://example.com/contact/): Phone, email and opening hours
## Optional
- [Blog](https://example.com/blog/): Seasonal advice for homeownersA few things to know about a file made this way:
- The server sends it as plain text. An Apache server with nothing configured for the file answers
200 text/plain, taking the type from the.txtending. - It is served whatever the permalinks are. The server finds the file before WordPress is asked, in the same way as a real
robots.txt. - It is yours to keep true. Nothing updates it when a page is renamed or removed.
- Switch a plugin's version off first. Yoast's specification says it will not overwrite an llms.txt that already exists and shows a warning instead, and tells you to disable its feature before creating your own.
- The links are ordinary pages. Version 2 of the proposal prefers links to markdown copies of pages. A WordPress site has none unless a plugin makes them, and Yoast's and Rank Math's documentation describe their files as lists of each page's ordinary address.
The file is public. Anyone can open it, so list nothing in it that you would not link from your menu.
Remove one a plugin made
Switch the feature off in the plugin that made it. These are the places each plugin's documentation gives.
| Plugin | Where the switch is |
|---|---|
| Yoast SEO | Yoast SEO, then Settings, then Site features. Under AI tools, the llms.txt toggle |
| Rank Math | Rank Math SEO, then Dashboard. The LLMS Txt module's toggle |
| All in One SEO | Sitemaps in the All in One SEO menu, then the LLMs.txt tab. The switches for llms.txt and llms-full.txt |
| SEOPress PRO | SEO, then PRO, then the llms.txt tab |
Then run the check from the section above again and expect a 404, or the home page while permalinks are Plain. If the address still answers, look in the site's root for a file named llms.txt and delete it there: a real file is served whatever a plugin's settings say. If a cache or a CDN stands in front of the site, clear it before deciding the file is still there.
Whether to bother
For a site that sells a service or runs a store, plainly:
- It costs little. A short file, or a switch in a plugin you already have.
- Nothing an engine has published says it changes whether you are quoted. Google says Search ignores it. The other four companies' pages say nothing about reading it. The one study with data found no relation.
- It is not access control. It does not block a crawler, allow one, or opt a site out of training.
- Its documented use is narrow. The proposal itself says the files are used most heavily for software documentation read by coding agents. A site with documentation for developers is the case it was written for.
So it is optional and low on the list. Leaving a plugin's file in place is harmless as long as what it lists is accurate. Paying for one as a way into AI answers is paying for something no engine has said it uses.
What does have documented standing is elsewhere. Whether an assistant's crawler can reach your pages at all is decided by robots.txt and by any CDN or host in front of the site, and blocking or allowing AI crawlers on WordPress covers that. What each engine says it looks for, and what is only guessed, is set out in generative engine optimization: what is documented, what was studied and what is guesswork.
This site serves an llms.txt of its own, at /llms.txt.
When to get help
- The address answers with something you cannot trace to a file or a plugin, or a file keeps coming back after you delete it. A one-time fix from WP Ministry covers one issue on one site and starts with a free diagnosis, which gives you a written cause and a fixed quote.
- You want the documented things checked for you: crawler access, indexing and what each engine's own reports show. That is the work described under AI search optimization.
Common questions
Does WordPress create an llms.txt file?
No. A fresh WordPress with no plugin active has no such file. The address answers with a 404, or with the home page while permalinks are set to Plain. The file appears when a plugin makes it or a person uploads it.
Does llms.txt help a site appear in Google's AI Overviews?
Google says no. Its guide to generative AI features in Search says Search does not use such files and that keeping one neither helps nor harms a site's visibility there. How to appear in Google's AI Overviews covers what Google says is needed.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers which addresses they may request, and crawlers that honor it follow it. llms.txt is a list of links with notes. It allows nothing and blocks nothing.
A tool says my site has no llms.txt. Is that a problem?
Not by Chrome's Lighthouse. Its documentation says a site with no file is marked not applicable, because providing one is optional, and that the audit flags only a server error at the address. A score from any other tool is that tool's own opinion.
What is llms-full.txt?
The proposal's page on llmstxt.org does not define it. It is a name some publishers give a second, fuller file. All in One SEO has a switch for one, which its documentation calls the full LLMs.txt file, and OpenAI's API documentation publishes one that it describes as a single-file Markdown export of its guides.
Can llms.txt stop AI companies from training on my content?
No. OpenAI, Anthropic and Perplexity each document robots.txt as the way to tell their crawlers what they may fetch. Blocking or allowing AI crawlers on WordPress covers the rules and their limits.
- GuideGenerative engine optimization (GEO): what is documented, what was studied and what is guesswork
- GuideHow to appear in Google AI Overviews: what Google requires and what keeps a WordPress page out
- GuideHow to block AI crawlers on WordPress, or let them in: robots.txt, Cloudflare, hosts and plugins
- GuideHow to stop AI training on your WordPress content and stay in AI search answers
- GuideAI bots slowing down your website: how to confirm it on WordPress and what to do, mildest first
- GuideBing Webmaster Tools for WordPress: setup, IndexNow, and what it means for ChatGPT and Copilot

