/ โ† AEO Fixes / Perplexity Invisibility
In this guide
  1. Using the sources list as evidence
  2. Perplexity's two user agents, as documented
  3. The direct URL test
  4. Confirming it in your logs
  5. Where Perplexity gets blocked, by platform
  6. The fix, step by step
  7. Unusual causes worth ruling out
  8. Keeping Perplexity access working
  9. Questions and answers

Using the sources list as evidence

Unlike assistants that sometimes answer without links, Perplexity is built around showing where an answer came from. Each reply carries numbered references and a sources view listing the pages it read. That list is your first diagnostic tool, because it tells you not just that you are missing but who took your place and what kind of page they offered.

Pattern in the sources listWhat it points to
Your domain never appears, even when you name your own businessPerplexity is probably unable to crawl you, or has not indexed you yet. Start with the access tests.
Directories and review pages about you appear, your own site does notYour pages are blocked or unreadable while third parties are not.
Another site carrying a copy of your article appears instead of youSyndication or scraping: the copy is reachable and yours is not, or the copy looks like the original.
Your page is listed as a source but the answer does not use itPerplexity can read you; the page is simply not the best match. That is an optimisation job.
You appear for your name but never for questions about your topicAccess is working. Move to the Perplexity optimisation guide.

Do this check in a logged-out or fresh session, and run each question two or three times. Perplexity's sources vary between runs, so a single missing appearance means little; a domain that never appears across a dozen runs is a real signal.

Perplexity's two user agents, as documented

Perplexity publishes a short page describing its crawlers. Read it before changing anything, because it is the authority on names and behaviour and it can change. At the time of writing it describes two agents with quite different jobs:

User agentWhat Perplexity says it is forHow Perplexity says robots.txt applies
PerplexityBotSurfacing and linking websites in Perplexity's search results. Perplexity states it is not used to crawl content for training AI foundation models.Respects robots.txt. Blocking it keeps your pages out of the index Perplexity uses to find sources.
Perplexity-UserFetching a page when a person's question or action in Perplexity needs it, for example when they ask about a specific address.Because a person requested the fetch, Perplexity says it generally ignores robots.txt rules.

Perplexity also publishes the IP address ranges both agents use, as JSON files linked from that documentation, and recommends that sites which want to be included allow both the user agent and those ranges through their firewall.

The split between the two agents is what makes the tests in this guide work. If PerplexityBot is blocked in robots.txt but nothing else is wrong, Perplexity-User can often still fetch a page when you ask about it directly. If both fail, the block is almost always at the firewall, CDN or server, because those layers do not care what robots.txt says.

โš ๏ธ In 2025 Cloudflare publicly criticised some of Perplexity's crawling behaviour, which Perplexity disputed, and bot-management products have since changed how they treat Perplexity traffic. If you use a CDN with bot controls, check how its current rules handle both Perplexity user agents rather than assuming they are let through.

The direct URL test

This is the quickest single test for Perplexity and it needs no server access. Pick a page with distinctive content, such as a price list or a staff page, and ask Perplexity something only that page can answer, including the full address:

Summarise https://www.example.co.uk/services/boiler-servicing
and tell me the price it gives for an annual service.

Then read the reply carefully:

Record the exact time you ran the test. You can then find that request in the access log in the next step and see the status code your server gave it.

๐Ÿ’ก Run the same test on a competitor's page that you know Perplexity cites. If theirs works and yours does not, you have ruled out a problem on Perplexity's side.

Confirming it in your logs

The direct URL test tells you something failed; the log tells you what. Search around the time of the test for both agents and look at the status codes:

# Every Perplexity request, with status and path
grep -E "PerplexityBot|Perplexity-User" access.log \
  | awk '{print $1, $9, $7}' | sort | uniq -c | sort -rn | head -40

The first field is the client IP address. Compare a few of them with the ranges in Perplexity's published JSON files. Genuine requests from those ranges receiving 403, 429 or 503 responses are your block. Requests that claim to be PerplexityBot from addresses outside those ranges are impostors and can be treated like any other unknown bot.

If the log shows no Perplexity request at all at the moment you ran the direct URL test, the request never reached your server. Look at the event log of your CDN or hosting firewall for that minute; the block will be recorded there, often with the name of the rule that fired.

A short Python check saves comparing addresses by hand. Download the JSON file named in Perplexity's documentation and test an address from your log:

import ipaddress, json
ranges = json.load(open("perplexitybot.json"))
nets = [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix"))
        for p in ranges["prefixes"]]
ip = ipaddress.ip_address("203.0.113.25")  # replace with the address from your log
print(any(ip in n for n in nets))

The keys follow the layout that Google and OpenAI use for their own range files. Open the file you downloaded and adjust the names if Perplexity's layout differs.

Where Perplexity gets blocked, by platform

PlatformWhere to lookWhat to change
Cloudflare and similar CDNsBot management, AI crawler controls, WAF custom rules, rate limiting rules and the security events logAllow both Perplexity agents from their published ranges; turn off blanket AI-crawler blocking if it catches them.
WordPressSecurity plugin firewall and blocked-traffic views; SEO plugin robots.txt editor; any bot-blocking pluginWhitelist the agents in the security plugin; remove Perplexity rules you did not intend.
ShopifyThe theme's robots.txt.liquid template; apps that add bot protection; store passwordRemove unwanted Disallow rules; make sure the store is not password-protected.
Managed hosts (Plesk, cPanel)ModSecurity rules, fail2ban jails, .htaccess user agent blocksCheck the server's own firewall logs for the IP; remove user agent blocks copied from "block AI bots" lists.
Static hosts and frameworksPlatform firewall or bot protection settings; the robots file your production build servesConfirm the deployed robots.txt, not the one in your repository.

Pay particular attention to "block AI bots" lists pasted into .htaccess or server configuration. They circulate widely, often include both Perplexity agents alongside training crawlers, and are easy to forget once added.

# .htaccess: an inherited rule that silently blocks Perplexity
RewriteCond %{HTTP_USER_AGENT} (GPTBot|PerplexityBot|Perplexity-User|ClaudeBot) [NC]
RewriteRule .* - [F,L]

If you want to keep blocking training crawlers, edit the list to remove only the agents that you want to answer questions about your business, rather than deleting the whole rule.

The fix, step by step

Step 1

Search the sources list for your own domain

Ask Perplexity the questions your customers ask, plus one that names your business, in a fresh session, and repeat each two or three times. Open the sources view each time and note whether your domain appears, which sites appear instead, and whether any of them carry copies of your content. Load the same questions into the AI Visibility Tracker so you have a dated baseline to compare against after the fix.

Step 2

Ask Perplexity to read one of your URLs

Paste the full address of a page with distinctive content and ask Perplexity to summarise it and quote a specific detail, such as a price. Note the exact time. If it quotes the detail correctly, fetching works and the problem lies with robots.txt, discovery or competitiveness. If it cannot access the page or gets the content wrong, something is stopping or starving the fetch, and the next steps will show which layer.

Step 3

Check robots.txt for PerplexityBot and wildcard rules

Open the live robots.txt on every host name the site uses. Look for a group naming PerplexityBot with a Disallow rule, and for a wildcard group that disallows everything, which applies when no named group exists. If you want to appear in Perplexity, add a User-agent: PerplexityBot group with Allow: /. Remember that Perplexity-User generally ignores robots.txt, so this rule mainly affects indexing.

Step 4

Verify Perplexity requests in the log against its IP ranges

Search the access log for both Perplexity user agents, including the moment of your direct URL test. Check the status codes they received and compare the client addresses with the IP ranges Perplexity publishes. Genuine requests receiving 403, 429 or 503 identify a server or plugin block. No request at all at the test time means it was stopped earlier, so open your CDN or host firewall event log for that minute.

Step 5

Let Perplexity through the firewall and CDN

Remove or edit whichever rule the logs identified. In a CDN, allow both Perplexity agents from their published IP ranges and switch off blanket AI-crawler blocking if it catches them. In WordPress security plugins, ModSecurity, fail2ban or .htaccess, remove Perplexity from inherited block lists while keeping any training crawlers you want to exclude. Exempt the published ranges from rate limits so repeated fetches are not treated as an attack.

Step 6

Serve the answer without scripts, logins or walls

Fetch the page with curl and confirm that the key facts are in the raw HTML. Move prices, opening times and service details out of images, scripts and tabs that only load on click. Make sure cookie banners overlay the page rather than replacing its content, and that no age check, login or regional redirect stands in front of public information. If a syndicated copy outranks you, ask the partner to add a canonical link to your original.

Step 7

Recheck the sources list after a week

Repeat the direct URL test first; it should now quote your page correctly. Confirm in the access log that PerplexityBot is receiving status 200 on your important pages. Then rerun the baseline questions in the AI Visibility Tracker and by hand, several times each, and compare the sources lists with the ones you recorded in step 1. Keep checking for a few weeks, because indexing and citation take time.

Unusual causes worth ruling out

Keeping Perplexity access working

Perplexity access tends to break for the same reason it broke the first time: a security tool or a copied rule. Add a monthly check to whoever looks after the site:

  1. Run the direct URL test on one key page and confirm it quotes the correct detail.
  2. Count PerplexityBot requests and their status codes in the last month's log.
  3. Re-download Perplexity's IP ranges if you allow-list by address, because ranges change.
  4. Read the robots.txt served in production after any deployment or plugin update.
  5. Rerun a fixed question set in the AI Visibility Tracker and compare with last month.

The same checks for every AI assistant, rather than Perplexity alone, are in fixing robots.txt AI blocks.

Questions and answers

1. Does Perplexity respect robots.txt?

Perplexity's documentation says PerplexityBot, the crawler it uses to surface and link websites in search results, respects robots.txt. It also says Perplexity-User, which fetches pages when a person asks for them, generally ignores robots.txt because the fetch was requested by a user. So a robots.txt rule mainly controls whether your pages are indexed for Perplexity search. If you want a complete block, you would need a firewall rule, and if you want to be included, allow both agents.

2. Is PerplexityBot used to train AI models?

Perplexity states that PerplexityBot is used to surface and link websites in its search results and is not used to crawl content for training AI foundation models. That is the provider's own description, and it is the best information available. If you are deciding whether to allow it, the practical trade-off is that blocking PerplexityBot keeps your pages out of Perplexity's search index, which means your site is unlikely to appear as a source in its answers.

3. How can I tell whether a PerplexityBot request is genuine?

Check the IP address, not just the user agent. Anyone can send a request that claims to be PerplexityBot. Perplexity publishes the IP ranges used by its agents as JSON files linked from its crawler documentation. Compare the address in your log with those ranges, either by hand or with a short script. Requests from outside the published ranges are not from Perplexity, and you can treat them like any other unidentified bot on your site.

4. Perplexity can summarise my page when I paste the link, so why am I never cited?

That result shows fetching works for user-requested visits, which Perplexity says generally ignore robots.txt. The next thing to check is whether PerplexityBot itself is disallowed in robots.txt, which would keep your pages out of the index it searches. If robots.txt allows it and your logs show it crawling successfully, the pages are reachable and the issue is that other sources answer the questions better, which is covered in the Perplexity optimisation guide.

5. Why does Perplexity cite a copy of my article on another site?

Usually because the copy was easier to reach or looked like the original. If your own page is blocked, slow or thin in its raw HTML, the copy wins by default. Unblock and speed up your page first. For legitimate syndication partners, ask them to add a canonical link pointing to your original and to publish after you. For unauthorised copies, contact the site or its host. Keep the date on your original page clear and accurate.

6. How long after unblocking will my site show up in Perplexity?

Perplexity does not publish a timescale for inclusion. Crawling resumes once the block is removed, but pages then need to be recrawled and judged useful for particular questions. Check the direct URL test straight away, since that should work immediately if the firewall was the problem. Then watch the access log for PerplexityBot receiving status 200 over the next week, and repeat your question set over several weeks to see whether your domain starts to appear in the sources.

7. Should I block Perplexity-User if I block PerplexityBot?

That depends on why you are blocking. If your aim is to keep your content out of Perplexity entirely, you would need to block both, and since Perplexity says the user agent generally ignores robots.txt, that means a firewall rule. If your aim is simply to reduce crawling load, blocking user-requested fetches may frustrate people who are specifically trying to read your page through Perplexity. Decide based on your business goals and write the policy down.

Check your Perplexity visibility

The AI Visibility Tracker asks AI assistants the questions your customers ask and records whether your brand is cited, mentioned, recommended or missing, so you can confirm Perplexity is ignoring you before you start and see when that changes.

Open AEO Fixes โ†’
๐Ÿ’ฌ Got a problem?
Related Guides: All AEO Fixes  ·  Optimising for Perplexity  ·  Fixing robots.txt AI Blocks  ·  Fixing AI Crawler Access  ·  Learning Hub

About aiwebpageseo

aiwebpageseo.com is a data-driven SEO and AEO (Answer Engine Optimisation) platform providing a free suite of technical SEO tools. Rather than relying on AI-theorised assumptions, the platform analyses live URL performance, delivering objective diagnostics, page speed metrics, CLS debugging, and site crawl data alongside actionable technical tutorials.