Most brands are debating whether to create an llms.txt file. A smaller group is also working on AGENTS.md. Almost nobody has heard of ai-catalog.json.
By the end of 2026, all three will matter — and they serve completely different purposes at completely different moments in an AI agent’s workflow. If you’re only implementing one, you’re optimizing for one layer of a three-layer system.
Here’s how the full stack works, when each layer fires, and what to prioritize right now.
Why AI Agents Need a Discovery Stack at All
Traditional search engines discover you through crawling and indexing. AI agents discover you through a different sequence entirely.
Research published in mid-2026 found that agents only open approximately 6% of the domains surfaced in a web search result. They’re not browsing — they’re selecting. And they’re selecting based on signals that predate the actual content visit: what’s in their training data, what registries say about you, and what your site tells them before they read a single page.
There’s also a staleness problem. As of June 2026, AI agent knowledge is approximately 5 months stale on average. The agent making decisions about your brand today may be working from information that was current in January. This makes real-time discoverability infrastructure — not just training data inclusion — essential.
The three-layer agentic discovery stack addresses these problems at three different trigger points.
Layer 1: AGENTS.md — The Environment Layer (Fires First)
When it fires: At the very start of an agent session, before any web browsing or retrieval occurs.
What it does: AGENTS.md, introduced by Anthropic, is a plaintext or markdown file placed at your repository root (for software projects) or accessible domain root. When an AI agent begins a session, it reads environment-level signals first — before querying anything else. AGENTS.md is one of those signals.
Think of it as the briefing document an agent reads before it starts working. It tells the agent what tools are available, how to interact with your system, what workflows exist, and what the agent should prioritize.
Why it matters: Experiments documented in 2026 found that the presence of a well-written AGENTS.md produced a “100% selection flip” in tool selection — meaning agents that would have chosen a competitor tool or alternative workflow consistently switched to the documented option when AGENTS.md was present. That’s not a marginal improvement. That’s complete redirection of agent behavior at the session level.
Who should implement it: Any company with an API, developer tool, plugin, or MCP server. If an agent might choose between your product and a competitor’s during a session, AGENTS.md is your first line of influence.
What to include: Tool descriptions, API endpoints, authentication patterns, example use cases, rate limits, and any behavioral guidance for agents interacting with your system. Write it for the agent, not for a human developer.
Layer 2: llms.txt — The Retrieval Layer (Fires When Agents Browse)
When it fires: When an agent visits your domain during a browsing or retrieval step.
What it does: llms.txt, proposed by Answer.AI’s Jeremy Howard, is a structured plaintext file at /llms.txt (with an optional /llms-full.txt variant) that tells language models what your site contains, how it’s structured, and which pages are most relevant for AI consumption.
It’s the difference between an agent having to infer your site’s structure from raw HTML versus being handed a map. When an agent lands on your domain, llms.txt is one of the first things it checks.
Why it matters: Analysis of llms.txt implementations found a ρ=−0.54 correlation between file staleness and retrieval quality. In practical terms: the older your llms.txt file, the worse your content performs in AI retrieval. Sites that updated their llms.txt monthly saw dramatically higher citation rates than those that set it and forgot it.
This is a maintenance story, not just an implementation story. A stale llms.txt is worse than a current one — potentially worse than nothing, because it actively misdirects agent attention.
Who should implement it: Every brand with web content that agents might retrieve. This is the most broadly applicable layer — it matters for SaaS, media, e-commerce, B2B services, and anyone doing GEO.
What to include: Site overview, primary topic areas, key pages with descriptions, content update frequency, and direct links to the most AI-relevant content. Keep it under 2,000 tokens for the base file; use llms-full.txt for comprehensive coverage.
Layer 3: ai-catalog.json — The Registry Layer (Fires When Agents Query Directories)
When it fires: When an agent is looking for services, tools, or capabilities to fulfill a task — specifically when querying a registry or directory of AI-capable endpoints.
What it does: On June 17, 2026, Google, Microsoft, and Hugging Face jointly announced the Agent Resource Descriptor (ARD) standard: a structured JSON file at /.well-known/ai-catalog.json that registers your domain’s AI capabilities in a machine-readable format.
If AGENTS.md is a briefing document and llms.txt is a site map, ai-catalog.json is your listing in the Yellow Pages for AI agents. When an agent needs to find a payment processor, a data provider, or a specific API capability, it can query a registry rather than browsing the open web. The ARD is what gets you into that registry.
Why it matters: This is the newest layer and the one with the most leverage for commercial discovery. As AI agents increasingly operate as autonomous buyers and task-executors rather than research tools, the ability to be found by agents that are actively looking for services becomes commercially critical.
The three-org backing (Google, Microsoft, Hugging Face) gives this standard significant adoption momentum. Expect it to become a ranking signal in agent-initiated searches within 6–12 months.
Who should implement it: Any business offering a service, API, or capability that another business or individual might hire an AI agent to find. This is the most commercially oriented layer — it’s about being discovered by agents on a buying mission.
What to include: Service descriptions, capability categories, pricing tiers, API documentation links, authentication methods, and use-case descriptions. The spec follows structured schema patterns similar to schema.org.
How the Three Layers Interact
These aren’t competing standards — they’re sequential layers in a single workflow:
- Agent starts a session → reads environment signals including
AGENTS.mdif working in a repository or documented environment - Agent needs information → browses the web, reads
llms.txton domains it visits to understand content structure - Agent needs a service or tool → queries a registry, finds you via
ai-catalog.jsonif you’re listed
A brand that has all three layers implemented is present at every decision point in an AI agent’s workflow. A brand with only llms.txt is present only during retrieval — invisible at session start and invisible in service registries.
There’s also a fourth layer worth mentioning: MCP (Model Context Protocol), sometimes called “USB-C for AI.” Anthropic donated MCP to the Agentic AI Foundation (under the Linux Foundation) in December 2025. MCP governs how agents actually connect to and invoke tools in real time. If AGENTS.md describes your tool, MCP is what enables the agent to use it. For brands building APIs or integrations, MCP compatibility is what closes the loop from discovery to execution.
What to Implement First
If you’re starting from scratch, prioritize based on your business model:
- Developer tools / APIs / SaaS: Start with
AGENTS.md. The session-start influence is your highest-leverage moment — a 100% selection flip outweighs any retrieval optimization. - Content-heavy brands / publishers: Start with
llms.txtand commit to monthly updates. Staleness is your biggest risk. - Service businesses / B2B: Implement
ai-catalog.jsonnow, before the registry ecosystem matures. Early listings have first-mover advantage in directory-style agent queries. - Enterprise with resources: All three, plus MCP compatibility if you have API endpoints. The full stack compounds — each layer reinforces the others.
The underlying principle is the same across all three: AI agents are not waiting to be found. They’re selecting from signals that exist before any browsing happens. The brands that structure those signals — in the right format, at the right location, updated on the right schedule — are the ones that get selected.
The Bottom Line
The debate over “should we do llms.txt” misses the larger question. There are now three standardized layers for agentic discoverability, and they serve different purposes at different moments. Implementing one is better than nothing. Implementing all three means you’re present from session start through service discovery — covering the full decision path of an AI agent working on behalf of a customer.
The stack is new. Most brands haven’t touched any of it. That’s the window.