How to Get Cited by AI Search Engines

What counts as being cited

When an assistant answers a question using a live web search, it can treat your business in one of three ways. It can cite you, showing a link to one of your pages as a source. It can mention you, naming your business in the text with no link. Or it can leave you out altogether. Only the first sends visitors to your site, and it is the one this guide is about.

Citations only appear when the assistant has actually fetched pages to write its answer. If it answers purely from what its model learned in training, there is usually nothing to cite, and you cannot change that answer quickly. So the practical goal is narrower than "be known by AI": it is to be one of the pages the assistant's search step finds, reads and decides is worth pointing to.

That search step behaves much like a search engine, which is good news. Most of what earns a citation is work you may already have done for Google: pages that can be crawled, are indexed, and answer a question clearly. The differences are in the detail, and they are where most sites go wrong.

What the AI companies have said about their sources

This is the part most advice gets wrong, so it is worth being precise. The table below sticks to what each company has said in its own documentation or announcements. Check their current help pages before making big decisions, because products change often.

What nobody has published

No AI company has published how it ranks or picks which of the retrieved pages to cite. Articles that claim an exact share of citations comes from a particular site or format are usually based on someone's own sample of prompts, which may not match your market, and the results can shift as the products change. Treat them as interesting, not as rules. The honest summary is: you can control whether you are eligible, and you can make your pages the clearest answer available, but the final choice is the assistant's.

Checking whether you are eligible at all

Before rewriting anything, prove that the assistants can reach and use your pages. Three quick checks catch most problems.

1. Can the crawler fetch the page?

Request a page while identifying as each crawler. This only tests rules based on the user agent name, not firewalls that check the crawler's IP address, but it catches robots.txt mistakes and simple bot blocks.

curl -s -o /dev/null -w "%{http_code}\n" -A "OAI-SearchBot" https://www.example.co.uk/services/
curl -s -o /dev/null -w "%{http_code}\n" -A "PerplexityBot" https://www.example.co.uk/services/
curl -s -o /dev/null -w "%{http_code}\n" -A "bingbot" https://www.example.co.uk/services/

A 200 is what you want. A 403, 429 or a challenge page means something between the crawler and your site is refusing it.

2. Are the real crawlers arriving?

The access log is the only proof that the genuine crawlers reach you. On a typical Linux server:

grep -E "OAI-SearchBot|ChatGPT-User|PerplexityBot|Claude-SearchBot|bingbot" access.log | awk '{print $9}' | sort | uniq -c

That counts the status codes each crawler received. Lots of 403 responses point at a firewall or security plugin. No lines at all can mean the crawler has not found the site, or that a CDN is blocking it before the request ever reaches your server, in which case check the CDN's own logs.

3. Is the page allowed to be quoted?

Search the page source for noindex, nosnippet, max-snippet and data-nosnippet, and check the X-Robots-Tag response header:

curl -sI https://www.example.co.uk/services/ | grep -i x-robots-tag

The fix, step by step

Things that will not get you cited

Because the field is new, a lot of confident advice is really guesswork. These are the common shortcuts, and why they do not work or carry risk.

• Adding an llms.txt file and waiting. The file is a proposal for giving language models a summary of a site. None of the major assistants has publicly confirmed using it to decide which sources to cite. It does no harm, but it is not a substitute for crawlable, indexed pages. See the llms.txt guide if you want one anyway.
• Special "AI" meta tags. There is no tag that asks an assistant to cite you. The tags that matter are the ones that can stop you being used: noindex and the snippet controls.
• Text hidden for bots. Serving crawlers different content from visitors is cloaking. Search engines treat it as spam, and it can remove pages from the very index the assistants draw on.
• Schema on its own. Structured data helps machines identify your business and your authors, but assistants quote visible text. A page with perfect markup and a vague answer will still lose to a clear one.
• Paying for placement in the answer. There is no published way to buy a place among the cited sources. Where assistants show adverts, they are labelled and kept apart from the sources.
• Mass-produced question pages. Hundreds of near-identical pages, one per question, look like doorway pages to search engines and rarely beat a single thorough page.

Platform notes

WordPress

Check Settings, Reading and make sure "Discourage search engines from indexing this site" is unticked; it is often left on after a site moves from staging. SEO plugins such as Yoast and Rank Math can set noindex or snippet limits per post type, so check the defaults for pages and posts, not just single items. Security plugins with bot blocking or rate limiting can refuse AI crawlers; look for their names in the plugin's blocked-request log.

Shopify

Shopify generates robots.txt from a template, robots.txt.liquid, which you can edit in the theme code. Most stores do not block AI crawlers by default, but apps and past edits can. Product and collection descriptions are often too short to answer a question; a proper guide page or a longer collection introduction gives the assistant something to quote.

Cloudflare and other CDNs

Cloudflare offers settings to block AI crawlers, and in 2025 it began blocking them by default for many newly added domains. Check the AI crawler and bot settings in your dashboard and decide deliberately which crawlers to allow. Other CDNs and hosts have similar bot management features that can block crawlers without any change to robots.txt.

JavaScript frameworks

If your site is built with React, Vue or a similar framework and renders content in the browser, many crawlers will see an almost empty page. Use server-side rendering or static generation for any page you want quoted, and confirm the answer text appears in the output of curl, not just in the browser.

How long it takes, and what to expect

Fixing a blocked crawler or a stray nosnippet can show results as soon as the page is recrawled, which may take days or a few weeks. Being indexed in Bing for the first time can take longer if the site has never been submitted there. Content changes take longer again, because the assistant has to recrawl the page and then prefer it over sources it already uses.

Expect uneven progress. You may be cited for some questions and not others, cited in one assistant and not another, and cited one day but not the next. That is normal behaviour for systems that generate every answer fresh, and it is why measuring a fixed set of questions over time matters more than any single result.

Keeping it working

• Re-check access after infrastructure changes. A new CDN, firewall rule, host or security plugin is the most common reason a site that was being cited suddenly stops.
• Watch the logs monthly. A simple count of requests from each AI crawler, with status codes, shows a block within weeks rather than months.
• Keep facts current. Prices, opening hours, service areas and staff change. Out-of-date pages are a common source of wrong answers about a business.
• Review lost questions quarterly. Look at the sources that are cited instead of you and ask what they give the reader that your page does not.

For the deeper fixes, see fixing low citation frequency for sites that are cited only occasionally, and fixing content structure for rewriting pages so answers can be lifted cleanly.