Sixteen checks, all passing, and it is worth being exact about what that does and does not mean. Every check on this report is a test of presence or access. Does the file exist at the path a client would look for. Does the header get sent. Does robots.txt permit the named crawler by name. These are binary, verifiable with curl, and reproducible by anyone.

None of them is a ranking factor, and none of them is a published citation factor for any assistant. No vendor — including this one — has access to the retrieval logic inside ChatGPT, Claude, Perplexity, Copilot or Google's AI Overviews, and no operator publishes it. What a perfect score means is that an automated client arriving at your site finds what it expects, is not turned away, and does not have to guess. That is a precondition for being used. It is not a promise that you will be.

This matters because the failure mode of agent-readiness advice is inflation: a checklist becomes a scorecard, the scorecard becomes a growth claim, and the growth claim cannot be substantiated. Read this report as a list of obstacles you have removed, and put your effort into the content behind the doors you have just unlocked.

The four discovery files sit under the site root or /.well-known/ so that a client can find them without being told where to look. They differ in maturity, and conflating them is how sites end up publishing files that describe capabilities they do not have.

llms.txt is a community proposal, not a standard published by any assistant operator. It is a Markdown file listing your most useful pages with brief descriptions, so a client with a limited context window can be pointed at the good material instead of crawling everything. Adoption is partial and its effect on citation is unproven. The strongest honest case for it is that it costs almost nothing and forces a genuinely useful editorial decision about which of your pages matter. Do not expect it to move anything on its own.

The API catalogue at /.well-known/api-catalog is defined by RFC 9727 and tells a client where your API descriptions live, so that it can discover your endpoints without a human reading your docs site. This is only worth publishing if you have an API. A catalogue pointing at nothing is a broken promise, and a client that follows it and fails has learned something worse about you than if the file had never existed.

The MCP server card advertises tools an assistant can call on a user's behalf — the capabilities you expose and how to invoke them. The A2A agent card describes an agent that other agents can interact with directly, for workflows with no human in the loop at each step. Both are only meaningful if the machinery behind them is live, authenticated and rate-limited. The card is the advertisement; the endpoint is the product.

The rule that governs all four: publish a discovery file only when the thing it describes is real and works. There is no partial credit for a well-formed file that points at a 404.

Four crawler checks pass here, and they are the only ones on the list where failure is catastrophic rather than merely suboptimal. If a named crawler cannot fetch your pages, nothing else on this report can compensate. But the four rows also carry the field's most persistent confusions, and it is worth stating each correction plainly.

GPTBot allowed means OpenAI may fetch your content for model training. It does not govern whether ChatGPT can cite you: that path uses OAI-SearchBot for search-and-cite and ChatGPT-User for user-initiated fetches. Blocking GPTBot removes you from a training corpus and nothing else. This is a content-licensing decision, and it is entirely reasonable to disallow training while allowing retrieval — but you have to name the agents separately to do it.

Google-Extended allowed means your content may be used to train Gemini. It has no effect on Google Search, on Googlebot's crawling, or on eligibility for AI Overviews, which are served through ordinary Googlebot access. A publisher can disallow Google-Extended and lose nothing in Search. Allowing it buys nothing in Search either. The row on this checklist is a training preference, and treating it as a visibility lever is the most common misreading of the whole audit.

ClaudeBot allowed and PerplexityBot allowed are Anthropic's and Perplexity's crawlers respectively. Both are retrieval paths rather than pure training crawlers, so a block here has a direct cost in reachability.

The critical caveat: robots.txt is a request, honoured by operators who choose to honour it, and it is not an access control. It also tells you nothing about what your infrastructure is doing. A permissive robots.txt and a WAF rule that drops requests matching the string "bot" will produce a passing check and zero crawler traffic. The only way to know which is true is to read your access logs and confirm the requests arrived and returned 200. Permission is not reachability.

Three of the sixteen checks are security-header hygiene, and the honest framing is that they belong on this list for correctness rather than for any effect on how machines read you. No assistant inspects your Content-Security-Policy before deciding whether to cite a page.

HSTS tells a browser to use HTTPS for your host for a stated period, which closes the window in which an initial plaintext request can be intercepted and downgraded. Content-Security-Policy constrains which scripts, styles and frames may execute, which is the primary defence against injected script — worth noting that CSP has no directive that disables document.write; Chrome intervenes against parser-inserted scripts on slow connections, which is a different mechanism entirely and frequently confused with CSP. The absence of X-XSS-Protection is a pass because that header is deprecated and its legacy filters could themselves be exploited; modern protection comes from CSP.

The Link header (RFC 8288) is the odd one out in this group: it is not a security control but a discovery mechanism, pointing clients at related resources such as your llms.txt or API catalogue at the HTTP layer, before any HTML is parsed. It is genuinely useful to a client that never intends to parse your markup at all.

A passing readiness audit is the end of the infrastructure work and the beginning of the real work. Four things are worth doing next, and none of them is on this checklist.

Verify in the logs. Every check here tests configuration. Your access log tests reality. Confirm that ClaudeBot, GPTBot and PerplexityBot are actually arriving and receiving 200s. A silent WAF block produces a perfect readiness score and zero crawler traffic, and only the log will tell you.

Check what is in the served HTML. Most retrieval systems parse rather than render. Content that appears only after a client-side fetch resolves is, to them, content that does not exist. This is not on the sixteen checks and it matters more than most of them.

Make the answers worth taking. An assistant summarising your page needs self-contained, quotable prose that stands up when lifted out of context. Structured data helps a parser identify what it is reading, but there is no verified evidence that denser markup earns more citations — the defensible claim is only that JSON-LD in the served response reaches parsers and disambiguates. Substance first, markup second.

Re-run after every infrastructure change. Bot access and discovery files break silently, at deployment time, and nothing in your application will report it. A CDN migration, a new bot-management ruleset or a robots.txt template refresh can undo all sixteen of these overnight.