ai-catalog.json Explained: The robots.txt for AI Agents?

Diagram of an ai-catalog.json file in the centre linking MCP servers, agent cards and skills on one side to registries and AI agents on the other. — Illustration by Forge Vertical.

What is ai-catalog.json? It is a machine-readable file a website or company publishes at /.well-known/ai-catalog.json to list the AI-usable resources it offers, such as MCP servers, A2A agent cards and agent skills. AI agents and registries read it to discover what exists before they try to call anything.

// Chapter 01 — The problem it solves

Agents can call tools. Finding them is the hard part.

Protocols like MCP define how an AI agent invokes a tool once it knows the tool exists. They say nothing about how the agent finds that tool in the first place. Today that happens through hardcoded integrations, hand-edited config files and marketplaces. ai-catalog.json is the "before" step: discovery before invocation. The idea is the same as robots.txt and sitemap.xml for crawlers: a predictable URL on your own domain where a machine can ask, "what do you offer?"

// Chapter 02 — Who is behind it

A coalition of big names, and a draft that is still moving

The file is the publishing side of Agentic Resource Discovery (ARD), a draft standard. According to AI Weekly, eleven companies published it on 17 June 2026: Google and Microsoft with Salesforce, Snowflake, ServiceNow, Cisco, Databricks, GitHub, GoDaddy, Hugging Face and Nvidia. It is Apache 2.0 licensed and builds on an AI Catalog data model maintained by a Linux Foundation working group. The same report says Anthropic and OpenAI are not among the backers. InfoQ reported it on 14 July, and another guide dates Google's announcement to May. The sources disagree on the exact date, which tells you how new this is.

// Early implementers infoq.com

GitHub's Agent Finder in Copilot and Hugging Face's Discover Tool are reported as early users of ARD for runtime capability discovery.

// Status aiweekly.co

A v0.9 draft. A tracker reported that, as of mid-June, none of 39 sampled top sites published an ai-catalog.json.

// The spec ai-catalog.io

The AI Catalog data model, served at the well-known path, with an optional trust layer for identity and signatures.

// The working repo github.com/Agent-Card/ai-catalog

A working repository for the common AI Card standard, with MCP and A2A communities collaborating.

// Chapter 03 — What the file looks like

Small, plain JSON

Per the specification, a catalog needs only a specVersion and an entries array. A host block with a display name is optional. Each entry needs an identifier, a type (a media type string), and exactly one of url or inline data. Descriptions, tags, versions and a signed trust manifest are optional, and consumers must ignore fields they do not recognise.

// /.well-known/ai-catalog.json (example from the spec site)
{
  "specVersion": "1.0",
  "host": { "displayName": "Example Hub" },
  "entries": [
    {
      "identifier": "urn:air:example.org:mcp:weather",
      "type": "application/mcp-server-card+json",
      "url": "https://example.org/mcp/weather/server-card"
    }
  ]
}

Identifiers follow the pattern urn:air:{publisher}:{namespace}:{name}, where the publisher is your domain. Recommended types include MCP server cards, A2A agent cards, agent skills and nested catalogs.

// Chapter 04 — Where it fits

Five files, five jobs

If you already work on SEO, AEO and GEO, this is the new layer on top of what you have. It does a different job from each of the others.

robotsPermission

robots.txt: what crawlers may fetch

Access rules for crawlers, including AI crawlers. It says who may read, not what you offer.

sitemapPages

sitemap.xml: which pages exist

A list of URLs so search engines can find and refresh your pages.

llmsSummary

llms.txt: a plain-text guide for language models

A readable overview of who you are and which pages matter, so a model can understand your site quickly.

schemaMeaning

schema.org JSON-LD: what each page means

Structured data that tells machines what a page is about: your business, services, prices, FAQs and people.

catalogCallable

ai-catalog.json: what an agent can use

Lists tools and agents an AI can actually call, such as MCP servers and agent cards. The first four are about being read. This one is about being used.

// Chapter 05 — The reality check

Interesting is not the same as proven

// What is known

The facts so far

A published draft with large backers
Registries are meant to crawl catalogs and answer agents' plain-language queries
It sits above MCP and does not replace it
GitHub and Hugging Face are reported as early implementers
It is free and just a static file

// What is not known

The open questions

Whether major assistants will fetch small-site catalogs
Whether Anthropic or OpenAI will join or build their own
Whether it affects AI citations at all
How registries will rank who gets discovered
Whether the draft will change before it settles

One more caution from the people building it: Microsoft's Jennifer Marsman said ARD "doesn't replace authentication, authorization, governance, or organizational trust decisions." Discovery tells an agent a tool exists. It does not decide whether the agent should be allowed to use it. And as API Evangelist notes, roughly three-quarters of providers still do not publish comparable discovery files such as APIs.json or Agent Cards.

// Chapter 06 — Should you publish one?

Our answer: yes, as a cheap experiment

It costs almost nothing and it fits the way we work. We build infrastructure early and measure later. But it is not a replacement for the basics that already have evidence behind them: schema on every page, llms.txt and consistent content. We found no published evidence yet that an ai-catalog.json changes AI citations, so treat it as a bet on where discovery is going and not as a ranking trick.

We published one. It lives at forgevertical.com/.well-known/ai-catalog.json and lists six real resources: our llms.txt, the AGIA definition page, the founder page, the national website offer, the free responsive design tester and the sitemap. We do not yet run an MCP server or agent card, and when we do, they will be the first entries that match the spec's main categories. Our first draft also taught us something: we copied an identifier format from a third-party guide, and the spec's own site uses a different one. We corrected it. With a draft this young, check the official specification before you ship.

01Create

Write the JSON

Start with specVersion, a host display name and one entry per real resource. List nothing you do not actually offer.

02Place

Host it at the well-known path

Serve it at /.well-known/ai-catalog.json over HTTPS. On GitHub Pages, add an empty .nojekyll file or the dot-folder will not publish.

03Check

Validate and compare to the spec

Confirm the file parses, loads in a browser and matches the current specification at ai-catalog.io.

04Link

Point to it from llms.txt

Mention the catalog URL in your llms.txt so models that read it can find it. Then watch your server logs for who fetches it.

// Verdict ai-catalog.json is the first serious attempt at a robots.txt for AI agents. It is early, unproven and backed by companies with real reach. For a small business the sensible move is a short, accurate file today, schema and llms.txt as the foundation, and a watch on whether the major assistants ever start to read it.

// Questions answered

What is ai-catalog.json?

ai-catalog.json is a machine-readable file that a website or company publishes at /.well-known/ai-catalog.json to list the AI-usable resources it offers, such as MCP servers, A2A agent cards and agent skills. Agents and registries read it to discover what exists before they try to call anything.

Where does ai-catalog.json go?

It goes at the well-known path /.well-known/ai-catalog.json on your own domain, served over HTTPS. On GitHub Pages the dot-folder only publishes if the repository contains a .nojekyll file.

What is the difference between ai-catalog.json, llms.txt and robots.txt?

robots.txt tells crawlers what they may fetch, and sitemap.xml lists your pages. llms.txt is a plain-text guide to your content for language models. ai-catalog.json is different: it lists callable resources such as MCP servers and agent cards, so agents can find tools and not only read pages.

Who backs ai-catalog.json and the ARD standard?

Agentic Resource Discovery (ARD) was published in mid-2026 by a coalition led by Google and Microsoft with Salesforce, Snowflake, ServiceNow, Cisco, Databricks, GitHub, GoDaddy, Hugging Face and Nvidia. It is an Apache 2.0 draft built on an AI Catalog data model maintained by a Linux Foundation working group. Anthropic and OpenAI were reported as not among the backers.

Does ai-catalog.json help SEO or AI citations?

We found no published evidence yet. It is a discovery standard for agents and tools, not a ranking or citation signal, and early reports found almost no top sites publishing one. Schema markup, llms.txt and strong content remain the proven GEO basics.

Should a small business publish an ai-catalog.json file?

It costs almost nothing, so it is a reasonable experiment if you list only real resources. Treat it as an addition to your schema and llms.txt work, and check the official specification again before relying on it because the draft is still changing.

// Sources

Written by

Jarrit Hosking

Forge Vertical · Cape Town · October 21, 2026

← All articles Tesla flying car → Free quote →