Jeremy Howard published version 2 of the llms.txt specification on August 10, 2026 — the first significant revision since he originally proposed the format in September 2024. The update formalizes how AI agents discover and retrieve Markdown versions of your pages, adds HTTP header support, and removes ambiguity around the Optional section convention that caused inconsistent implementations in the original spec. If you already have an llms.txt file, the v2 update requires a minor change rather than a rebuild. If you’ve been debating whether to build one at all, this post covers both what’s new and the honest case for whether it’s worth your time.
What Changed in v2
The v2 spec makes four concrete changes to the original format. First, it adds formal link relations — a way for AI agents to discover llms.txt and associated Markdown files through HTTP headers and HTML link elements, rather than relying solely on file discovery at the root URL. A site can now signal the existence of its llms.txt via a Link header in HTTP responses, which means agents that don’t crawl the root path can still find the file. Second, v2 formalizes a second URL pattern for Markdown pages. The original spec assumed a single pattern for converting HTML page URLs to Markdown equivalents; v2 accommodates sites that publish Markdown under a different path structure and provides agents with a formal mechanism to locate them. Third, the Optional section — previously treated as a structured convention for lower-priority content that agents could skip to save context — has had its mechanical significance removed. It now functions as a labeling convention only, not as a parsing instruction agents are expected to honor. Fourth, the examples and background documentation on llmstxt.org have been updated to reflect real agent use patterns, specifically coding agents fetching documentation, rather than the speculative browsing scenarios in the original spec.
For sites with an existing implementation: the practical change is adding a Link header to your HTTP responses pointing to your llms.txt file. Most web servers and CMS platforms can add this header without code changes. For WordPress sites, a single filter in functions.php covers it, or any of the llms.txt plugins that have released v2-compatible updates.
Who Is Actually Reading llms.txt
Before making the case for or against building an llms.txt file, the crawl data tells a story worth reading carefully. A 90-day analysis of AI bot traffic across a monitored network found that out of over 500 million AI bot visits, only 408 targeted llms.txt directly. That’s not a rounding error — it reflects the current reality of which AI systems use the file and which ignore it.
The primary consumers of llms.txt today are coding agents: Claude, Cursor, Windsurf, and similar tools that need to understand a codebase or API documentation before generating code. These agents fetch llms.txt reliably when they encounter a developer-facing site. This explains why adoption has been fastest in developer tools, API documentation, and open-source project sites — categories where coding agents are the dominant traffic source.
Consumer AI search tools — ChatGPT, Perplexity, Google AI Mode — do not read llms.txt during query-response generation. Google’s official guidance on optimizing for AI Overviews and AI Mode, published May 15, 2026, does not mention llms.txt at all. Google did add an llms.txt audit to Lighthouse on May 5, 2026, under a new “agentic browsing” category — which signals forward intent — but that audit does not affect current AI Overview or AI Mode citation eligibility. Perplexity’s crawlers visit root paths during index builds, but there is no confirmed evidence they read llms.txt as a retrieval signal for consumer queries.
The Honest Investment Case
Overall adoption is at 10.13% across a SE Ranking study of 300,000 domains — low enough that llms.txt is still a differentiating signal rather than table stakes. The low adoption rate is partly an implementation friction issue (most CMS platforms don’t auto-generate it) and partly a “wait and see” posture from teams uncertain about the return.
The case for building one now rests on three arguments. First, the Lighthouse audit inclusion signals that Google is building toward using llms.txt for agentic browsing evaluation — sites that have a correct v2 implementation will be ahead of the change rather than retrofitting after the fact. Second, if your site has substantial developer or API documentation, coding agents are actively using the file today, and a well-structured llms.txt reduces the context load those agents need to understand your product — which translates directly to more accurate agent-generated recommendations and code samples. Third, the implementation cost is low relative to other GEO investments: a correctly built llms.txt for most sites takes two to four hours including Markdown page generation for key documentation URLs.
The case against prioritizing it over other GEO work: if your primary AI visibility gap is in ChatGPT, Perplexity, or Google AI Overviews responses to general queries, llms.txt is not the lever to pull. Structured content, answer-first formatting, FAQ schema, entity authority, and Reddit or LinkedIn presence will move those citation metrics faster than an llms.txt file will.
The v2 Implementation Structure
A correct v2 llms.txt follows this format at its root:
# [Site or Project Name]
> [One to three sentence summary of what the site covers and who it's for — written for an AI agent that has no prior context]
## [Primary Section: Core Documentation or Key Pages]
- [Page Title](https://example.com/page/): [One-sentence description of what this page covers and when an agent should read it]
- [Page Title](https://example.com/page-2/): [Description]
## [Secondary Section: Optional but Useful]
- [Page Title](https://example.com/supporting/): [Description]
## [llms-full.txt](https://example.com/llms-full.txt)
[Optional: Link to the full-text version containing complete page content rather than URL pointers]Three implementation details that most tutorials skip. First, the summary blockquote (the line starting with `>`) is the highest-signal text in the file — agents use it to determine whether to read further or stop. Write it as a direct, factual description: what the site is, what categories it covers, and what problems it helps solve. Promotional language in the summary reduces its utility for agent decision-making. Second, page descriptions should answer the implicit question “why would an agent fetch this page?” rather than describe the page’s content passively. “Overview of pricing tiers” is less useful than “read this to understand per-seat vs usage-based pricing options and which plan matches which scale of operation.” Third, for v2 compliance, add the HTTP Link header to your site’s responses: `Link: ; rel=”llms-txt”`. On Nginx, this is a single `add_header` directive in your server block.
Generating llms-full.txt
The llms-full.txt companion file — referenced in the v2 spec — contains the complete text of each page rather than URL pointers. Agents with large context windows can ingest the entire documentation set from a single file request, which is more efficient than fetching 40 individual pages. For sites with structured documentation, the full.txt file is often more valuable to coding agents than the index file.
The generation approach depends on your stack. For WordPress sites, the RowanWeb Markdown for AI plugin generates both files automatically and updates them on post publish. For static sites (Jekyll, Hugo, Astro), a build step that converts output HTML to Markdown and concatenates it into llms-full.txt covers both files. For custom stacks, a Node.js or Python script that fetches your sitemap, extracts content from each URL, converts to Markdown, and writes both output files handles the generation in under 50 lines of code. The file should be regenerated whenever substantial content changes — a weekly cron for most sites is sufficient.
Schema and llms.txt: Complementary, Not Competing
A May 2026 study reported by Search Engine Journal found that adding JSON-LD schema did not measurably increase AI citations for pages already visible in AI Overviews. The finding generated pushback from practitioners who’ve seen citation lifts from schema work — and the resolution is in the qualifier “already visible.” Schema’s primary citation lift comes from making pages eligible for AI retrieval in the first place, not from increasing citation frequency for pages already in the citation pool. For pages that are correctly indexed, well-structured, and already appearing in AI answers, adding more schema produces diminishing returns.
llms.txt and schema solve different problems. Schema tells AI systems what a page is about and what structured data it contains — this affects retrieval eligibility. llms.txt tells AI agents which pages exist and which are worth reading for which use cases — this affects discovery and navigation efficiency. A site with strong schema and no llms.txt is well-indexed but poorly discoverable by agentic systems. A site with llms.txt and weak schema may be discoverable but still excluded from citation pools for specific query types. Both are necessary; neither substitutes for the other.
Implementation Priority: Where llms.txt Fits in Your GEO Stack
If you’re building a GEO implementation from scratch and need to prioritize time, the sequence that produces the fastest citation improvement is: structured content and answer-first formatting first (2-4 weeks to impact), FAQ and Article schema second (2-4 weeks), entity establishment across third-party sources third (4-8 weeks), then llms.txt and Markdown page generation as a fourth layer (immediate impact for coding agent traffic; longer-horizon impact for consumer AI search as the ecosystem evolves toward agentic browsing). LinkedIn Article publishing and Reddit presence build in parallel across all stages for citation source diversity.
Teams with developer-facing products or API documentation should move llms.txt earlier in the sequence — to second or third place — because the immediate return from coding agent traffic is measurable and direct. A developer who asks Claude or Cursor about your API and gets an accurate, confident answer because the agent read your llms.txt is a concrete outcome. For non-developer products, the sequence above holds.
To see where your domain currently appears across ChatGPT, Perplexity, Google AI Overviews, and Claude — and which structural gaps are creating the biggest citation losses — run a free audit at ai-visibility.llmagnet.com.