Based on the data available at the end of 2026, publishing an llms.txt file does not measurably increase your visibility or your citations in ChatGPT, Perplexity or Google. I’m telling you up front because it’s the question almost nobody answers with data. llms.txt has been sold as the big lever for AI visibility, but the two serious studies that exist point the other way. It does have a real use case, and it’s important to understand it, but it isn’t the one you’ve been told about.
What llms.txt is and what problem it set out to solve
Jeremy Howard (co-founder of Answer.AI and fast.ai) published the proposal in September 2024. It starts from a real problem: web pages are built for people, with the content wrapped in navigation, ads and JavaScript, and turning that HTML into clean text is hard and imprecise for a model. On top of that, LLM context windows are still too small to process an entire site.
The proposed solution is a Markdown file at the root of the domain (/llms.txt) that gives a short overview of the site and links to clean versions of its key pages. One detail matters: it was designed for inference (an agent looking up information while it helps a user), not for training or for rankings. Ahrefs puts it bluntly: the “AI visibility” angle around llms.txt was added later by the SEO industry.
How to create an llms.txt file
The specification is minimal. The order is:
- An H1 with the site name. It’s the only required element.
- A blockquote (
>) with a short summary. - Optional paragraphs or lists with context.
- H2 sections with lists of links in the format
[name](url): note. - By convention, an “Optional” section with links the agent can skip if it’s short on context.
An example for a consultancy:
# Linking & Growing
> SEO/GEO consultancy in Valencia specialising in organic visibility and AI search engines.
## Services
- [Technical SEO audit](https://example.com/audit.md): scope, deliverables and timelines
- [GEO consulting](https://example.com/geo.md): methodology and case studies
## Optional
- [Blog](https://example.com/blog.md)The proposal asks for something almost every marketing implementation forgets: also publishing a .md version of every linked page. And there’s a sibling, llms-full.txt, which concatenates all the content into a single Markdown file so it can be pasted into a developer tool in one go.
In WordPress, it can be generated by Yoast (since June 2025, including the free version), Rank Math and the “Website LLMs.txt” plugin. A word of warning: generating it automatically doesn’t guarantee it’s done right. A real Lighthouse test on a file generated by Yoast reported that it didn’t follow the recommendations.
llms.txt is NOT robots.txt
This is the most common mistake. robots.txt controls access: it says which bots can crawl which paths. llms.txt is a curation suggestion: it doesn’t block, doesn’t authorise and doesn’t enforce anything. As Ahrefs notes, it neither controls nor blocks anything. And it doesn’t override robots.txt: if you block OAI-SearchBot, no llms.txt will get you into ChatGPT Search.
The key question: does it work?
This is where the data speaks clearly.
SE Ranking: no correlation with citations. They analysed close to 300,000 domains (November 2025). Only 10.13% had an llms.txt, and they found no correlation between having one and being cited by LLMs. Most revealing of all: when they removed the llms.txt variable, their predictive model performed better. In other words, today the file adds more noise than value.
Ahrefs: 97% of these files are never read. They reviewed the logs of 137,210 domains with traffic in May 2026. Of those publishing a valid llms.txt, 97% didn’t receive a single request for it all month. Among the remaining 3%, AI search retrieval bots (OAI-SearchBot, PerplexityBot) made just 1.1% of the requests. The figure that sums it all up: Slackbot requested more llms.txt files than PerplexityBot did.
Google: it’s in writing. Its official generative AI documentation (2026) lists llms.txt among the things you don’t need, and makes clear these files affect visibility neither negatively nor positively. John Mueller compared it to the old meta keywords tag, a self-declared, unverifiable signal, and Gary Illyes confirmed that Google doesn’t support it and has no plans to.
OpenAI, Anthropic and Perplexity. All three manage their bots’ access through robots.txt. None of them has officially stated that it reads third-party sites’ llms.txt files to decide what to cite. GPTBot downloading a file now and then doesn’t mean it uses it.
“I’ve had to debunk this in meetings: someone has read that llms.txt is the future of SEO and wants us to set it up right away. And I do set it up, because it takes five minutes and does no harm, but I make it clear it isn’t the lever they think it is. If an agency bills you for an ‘llms.txt strategy’ as a way to show up in ChatGPT, ask them for the logs and the before-and-after citations. They won’t have them, because that data doesn’t exist.” – Rafa Villaplana, SEO consultant and CEO of Linking & Growing
Where it does make sense: documentation for coding agents
The real use case is the original one, not the marketing one. The specification itself acknowledges it: llms.txt is used mainly in software documentation, where coding agents follow it. In Ahrefs’ data, Claude Code is the second bot that downloads the most llms.txt files, ahead of any AI search engine. Tools like Cursor let you give the agent the URL of a library’s llms.txt so it doesn’t rely on outdated training data.
In plain terms: if you sell software, an API or an SDK, llms.txt (with its .md versions) reduces errors and token usage for the agents your developer customers use. It’s a product and developer-experience advantage, not an SEO one.
What to do instead to gain visibility in AI
These are levers with evidence behind them:
- Extractable content with data and sources. The academic paper that coined the term GEO showed that adding statistics, quotes and references increased visibility in generative answers by up to 40% in its benchmark. Keyword stuffing didn’t work.
- Brand mentions. They correlate at 0.664 with visibility in AI Overviews, compared with 0.218 for backlinks (Ahrefs, 75,000 brands).
- Freshness. AI assistants cite URLs that are 25.7% newer than those in organic results. Update content thoroughly, not just the date.
- Bot accessibility. AI crawlers don’t render JavaScript: if your content depends on client-side JS, they can’t see it. Serve key content in HTML and check what GPTBot, OAI-SearchBot and PerplexityBot can actually see.
Verdict: should you set up an llms.txt in 2026?
- Yes, carefully, if you publish technical documentation, APIs or SDKs that developers query with agents. Do it with .md versions and validate it.
- Optional, with no expectations, if you’re an SME, an ecommerce store, a professional firm or a media outlet. Turning it on in Yoast or Rank Math takes minutes and Google doesn’t penalise it, but the proven benefit is zero.
- No, as a billable service or as a “GEO strategy”. That hour pays off far more spent reviewing your logs, serving your content in HTML and earning mentions in your industry’s media.
“My rule with clients is simple: I don’t charge you to set up an llms.txt, because that would mean charging you for something with no proven impact. I’ll switch it on if you want, and we spend the real time on what moves citations today. Being honest about the data is part of the service.” – Rafa Villaplana, SEO consultant and CEO of Linking & Growing
Frequently asked questions
Does llms.txt help you show up in ChatGPT or Google’s AI Overviews?
There’s no evidence that it helps. Two independent studies (SE Ranking on ~300,000 domains and Ahrefs on 137,210) found no relationship between having an llms.txt and getting more citations, and Google states in writing that its search engine ignores it.
So why is everyone talking about setting up an llms.txt?
Because the SEO industry added an “AI visibility” framing that the original proposal never had. The file was designed for coding agents to look up technical documentation, not to rank a marketing website.
Is llms.txt the same as robots.txt?
No. robots.txt controls which bots can crawl your site; llms.txt only suggests curated content and doesn’t block or authorise anything. If you block a bot in robots.txt, llms.txt doesn’t override it.
Will Google penalise me for having an llms.txt?
No. Google has made clear that these files affect visibility in Search neither positively nor negatively. You can keep it with no risk, but don’t expect any benefit.
When should I set one up?
If your business offers technical documentation, an API or an SDK that developers use with tools like Claude Code or Cursor. In that case agents do consume it, and it reduces errors and token usage.