Not Cited by Perplexity? Test PerplexityBot Access, Fix It
Perplexity lists its sources beside nearly every answer, which makes absence painfully obvious: you can scroll the list and see every site except yours. The upside is that the same list, together with Perplexity's published crawler documentation, makes the cause unusually easy to track down.
This guide walks through a set of tests that separate a robots.txt rule, a firewall that turns Perplexity away, a page Perplexity cannot read, and a copy of your content winning in your place, and then shows how to clear each one.
Using the sources list as evidence
Unlike assistants that sometimes answer without links, Perplexity is built around showing where an answer came from. Each reply carries numbered references and a sources view listing the pages it read. That list is your first diagnostic tool, because it tells you not just that you are missing but who took your place and what kind of page they offered.
| Pattern in the sources list | What it points to |
|---|---|
| Your domain never appears, even when you name your own business | Perplexity is probably unable to crawl you, or has not indexed you yet. Start with the access tests. |
| Directories and review pages about you appear, your own site does not | Your pages are blocked or unreadable while third parties are not. |
| Another site carrying a copy of your article appears instead of you | Syndication or scraping: the copy is reachable and yours is not, or the copy looks like the original. |
| Your page is listed as a source but the answer does not use it | Perplexity can read you; the page is simply not the best match. That is an optimisation job. |
| You appear for your name but never for questions about your topic | Access is working. Move to the Perplexity optimisation guide. |
Do this check in a logged-out or fresh session, and run each question two or three times. Perplexity's sources vary between runs, so a single missing appearance means little; a domain that never appears across a dozen runs is a real signal.
Perplexity's two user agents, as documented
Perplexity publishes a short page describing its crawlers. Read it before changing anything, because it is the authority on names and behaviour and it can change. At the time of writing it describes two agents with quite different jobs:
| User agent | What Perplexity says it is for | How Perplexity says robots.txt applies |
|---|---|---|
PerplexityBot | Surfacing and linking websites in Perplexity's search results. Perplexity states it is not used to crawl content for training AI foundation models. | Respects robots.txt. Blocking it keeps your pages out of the index Perplexity uses to find sources. |
Perplexity-User | Fetching a page when a person's question or action in Perplexity needs it, for example when they ask about a specific address. | Because a person requested the fetch, Perplexity says it generally ignores robots.txt rules. |
Perplexity also publishes the IP address ranges both agents use, as JSON files linked from that documentation, and recommends that sites which want to be included allow both the user agent and those ranges through their firewall.
The split between the two agents is what makes the tests in this guide work. If PerplexityBot is blocked in robots.txt but nothing else is wrong, Perplexity-User can often still fetch a page when you ask about it directly. If both fail, the block is almost always at the firewall, CDN or server, because those layers do not care what robots.txt says.
The direct URL test
This is the quickest single test for Perplexity and it needs no server access. Pick a page with distinctive content, such as a price list or a staff page, and ask Perplexity something only that page can answer, including the full address:
Summarise https://www.example.co.uk/services/boiler-servicing
and tell me the price it gives for an annual service.
Then read the reply carefully:
- It quotes the right price and lists your page as a source. The page can be fetched and read. If you still never appear for normal questions, the problem is either PerplexityBot being blocked in robots.txt, discovery, or competitiveness. Check robots.txt next.
- It says it cannot access the page, or summarises other sites instead. The fetch failed. Because Perplexity-User generally ignores robots.txt, a failure here points to a firewall, bot protection, geo-blocking, a login or a server error.
- It fetches the page but gets the content wrong or thin. The fetch worked but the HTML it received does not contain the text, usually because the content is loaded by JavaScript, hidden behind a consent screen, or inside an image.
Record the exact time you ran the test. You can then find that request in the access log in the next step and see the status code your server gave it.
Confirming it in your logs
The direct URL test tells you something failed; the log tells you what. Search around the time of the test for both agents and look at the status codes:
# Every Perplexity request, with status and path
grep -E "PerplexityBot|Perplexity-User" access.log \
| awk '{print $1, $9, $7}' | sort | uniq -c | sort -rn | head -40
The first field is the client IP address. Compare a few of them with the ranges in Perplexity's published JSON files. Genuine requests from those ranges receiving 403, 429 or 503 responses are your block. Requests that claim to be PerplexityBot from addresses outside those ranges are impostors and can be treated like any other unknown bot.
If the log shows no Perplexity request at all at the moment you ran the direct URL test, the request never reached your server. Look at the event log of your CDN or hosting firewall for that minute; the block will be recorded there, often with the name of the rule that fired.
A short Python check saves comparing addresses by hand. Download the JSON file named in Perplexity's documentation and test an address from your log:
import ipaddress, json
ranges = json.load(open("perplexitybot.json"))
nets = [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix"))
for p in ranges["prefixes"]]
ip = ipaddress.ip_address("203.0.113.25") # replace with the address from your log
print(any(ip in n for n in nets))
The keys follow the layout that Google and OpenAI use for their own range files. Open the file you downloaded and adjust the names if Perplexity's layout differs.
Where Perplexity gets blocked, by platform
| Platform | Where to look | What to change |
|---|---|---|
| Cloudflare and similar CDNs | Bot management, AI crawler controls, WAF custom rules, rate limiting rules and the security events log | Allow both Perplexity agents from their published ranges; turn off blanket AI-crawler blocking if it catches them. |
| WordPress | Security plugin firewall and blocked-traffic views; SEO plugin robots.txt editor; any bot-blocking plugin | Whitelist the agents in the security plugin; remove Perplexity rules you did not intend. |
| Shopify | The theme's robots.txt.liquid template; apps that add bot protection;
store password | Remove unwanted Disallow rules; make sure the store is not password-protected. |
| Managed hosts (Plesk, cPanel) | ModSecurity rules, fail2ban jails, .htaccess user agent blocks | Check the server's own firewall logs for the IP; remove user agent blocks copied from "block AI bots" lists. |
| Static hosts and frameworks | Platform firewall or bot protection settings; the robots file your production build serves | Confirm the deployed robots.txt, not the one in your repository. |
Pay particular attention to "block AI bots" lists pasted into .htaccess or server configuration.
They circulate widely, often include both Perplexity agents alongside training crawlers, and are easy
to forget once added.
# .htaccess: an inherited rule that silently blocks Perplexity
RewriteCond %{HTTP_USER_AGENT} (GPTBot|PerplexityBot|Perplexity-User|ClaudeBot) [NC]
RewriteRule .* - [F,L]
If you want to keep blocking training crawlers, edit the list to remove only the agents that you want to answer questions about your business, rather than deleting the whole rule.
The fix, step by step
Step 1
Search the sources list for your own domain
Ask Perplexity the questions your customers ask, plus one that names your business, in a fresh session, and repeat each two or three times. Open the sources view each time and note whether your domain appears, which sites appear instead, and whether any of them carry copies of your content. Load the same questions into the AI Visibility Tracker so you have a dated baseline to compare against after the fix.
Step 2
Ask Perplexity to read one of your URLs
Paste the full address of a page with distinctive content and ask Perplexity to summarise it and quote a specific detail, such as a price. Note the exact time. If it quotes the detail correctly, fetching works and the problem lies with robots.txt, discovery or competitiveness. If it cannot access the page or gets the content wrong, something is stopping or starving the fetch, and the next steps will show which layer.
Step 3
Check robots.txt for PerplexityBot and wildcard rules
Open the live robots.txt on every host name the site uses. Look for a group
naming PerplexityBot with a Disallow rule, and for a wildcard group that disallows everything,
which applies when no named group exists. If you want to appear in Perplexity, add a User-agent: PerplexityBot
group with Allow: /. Remember that Perplexity-User generally ignores robots.txt, so this
rule mainly affects indexing.
Step 4
Verify Perplexity requests in the log against its IP ranges
Search the access log for both Perplexity user agents, including the moment of your direct URL test. Check the status codes they received and compare the client addresses with the IP ranges Perplexity publishes. Genuine requests receiving 403, 429 or 503 identify a server or plugin block. No request at all at the test time means it was stopped earlier, so open your CDN or host firewall event log for that minute.
Step 5
Let Perplexity through the firewall and CDN
Remove or edit whichever rule the logs identified. In a CDN, allow both Perplexity agents
from their published IP ranges and switch off blanket AI-crawler blocking if it catches them. In WordPress
security plugins, ModSecurity, fail2ban or .htaccess, remove Perplexity from inherited block
lists while keeping any training crawlers you want to exclude. Exempt the published ranges from rate
limits so repeated fetches are not treated as an attack.
Step 6
Serve the answer without scripts, logins or walls
Fetch the page with curl and confirm that the key facts are in the
raw HTML. Move prices, opening times and service details out of images, scripts and tabs that only load
on click. Make sure cookie banners overlay the page rather than replacing its content, and that no age
check, login or regional redirect stands in front of public information. If a syndicated copy outranks
you, ask the partner to add a canonical link to your original.
Step 7
Recheck the sources list after a week
Repeat the direct URL test first; it should now quote your page correctly. Confirm in the access log that PerplexityBot is receiving status 200 on your important pages. Then rerun the baseline questions in the AI Visibility Tracker and by hand, several times each, and compare the sources lists with the ones you recorded in step 1. Keep checking for a few weeks, because indexing and citation take time.
Unusual causes worth ruling out
- A copy of your content wins. If a partner republishes your articles, or a scraper copies them, Perplexity may cite the copy. Ask legitimate partners to use a canonical link to your original and to publish after you; report scrapers to their host. Make sure your own page is the fastest to load and the clearest to read.
- You recently moved domain or restructured URLs. Old addresses that redirect through several hops, or that return 404, lose their place. Keep redirects to a single hop and update your sitemap so the new addresses are discovered.
- Your server is slow or times out under load. A fetch that takes too long may be abandoned. Check the log for requests from Perplexity that took several seconds or ended in 5xx errors, and fix the performance problem.
- A shared name. If another business uses a similar name, Perplexity may be finding the other one. Include your town, trade and web address consistently wherever you are listed.
- You have chosen to block it. Blocking Perplexity is a legitimate choice. Just be aware that blocking PerplexityBot in robots.txt alone may not stop user-requested fetches, according to Perplexity's own description, so a firewall rule is the only complete block.
Keeping Perplexity access working
Perplexity access tends to break for the same reason it broke the first time: a security tool or a copied rule. Add a monthly check to whoever looks after the site:
- Run the direct URL test on one key page and confirm it quotes the correct detail.
- Count PerplexityBot requests and their status codes in the last month's log.
- Re-download Perplexity's IP ranges if you allow-list by address, because ranges change.
- Read the robots.txt served in production after any deployment or plugin update.
- Rerun a fixed question set in the AI Visibility Tracker and compare with last month.
The same checks for every AI assistant, rather than Perplexity alone, are in fixing robots.txt AI blocks.
1. Does Perplexity respect robots.txt?
Perplexity's documentation says PerplexityBot, the crawler it uses to surface and link websites in search results, respects robots.txt. It also says Perplexity-User, which fetches pages when a person asks for them, generally ignores robots.txt because the fetch was requested by a user. So a robots.txt rule mainly controls whether your pages are indexed for Perplexity search. If you want a complete block, you would need a firewall rule, and if you want to be included, allow both agents.
2. Is PerplexityBot used to train AI models?
Perplexity states that PerplexityBot is used to surface and link websites in its search results and is not used to crawl content for training AI foundation models. That is the provider's own description, and it is the best information available. If you are deciding whether to allow it, the practical trade-off is that blocking PerplexityBot keeps your pages out of Perplexity's search index, which means your site is unlikely to appear as a source in its answers.
3. How can I tell whether a PerplexityBot request is genuine?
Check the IP address, not just the user agent. Anyone can send a request that claims to be PerplexityBot. Perplexity publishes the IP ranges used by its agents as JSON files linked from its crawler documentation. Compare the address in your log with those ranges, either by hand or with a short script. Requests from outside the published ranges are not from Perplexity, and you can treat them like any other unidentified bot on your site.
4. Perplexity can summarise my page when I paste the link, so why am I never cited?
That result shows fetching works for user-requested visits, which Perplexity says generally ignore robots.txt. The next thing to check is whether PerplexityBot itself is disallowed in robots.txt, which would keep your pages out of the index it searches. If robots.txt allows it and your logs show it crawling successfully, the pages are reachable and the issue is that other sources answer the questions better, which is covered in the Perplexity optimisation guide.
5. Why does Perplexity cite a copy of my article on another site?
Usually because the copy was easier to reach or looked like the original. If your own page is blocked, slow or thin in its raw HTML, the copy wins by default. Unblock and speed up your page first. For legitimate syndication partners, ask them to add a canonical link pointing to your original and to publish after you. For unauthorised copies, contact the site or its host. Keep the date on your original page clear and accurate.
6. How long after unblocking will my site show up in Perplexity?
Perplexity does not publish a timescale for inclusion. Crawling resumes once the block is removed, but pages then need to be recrawled and judged useful for particular questions. Check the direct URL test straight away, since that should work immediately if the firewall was the problem. Then watch the access log for PerplexityBot receiving status 200 over the next week, and repeat your question set over several weeks to see whether your domain starts to appear in the sources.
7. Should I block Perplexity-User if I block PerplexityBot?
That depends on why you are blocking. If your aim is to keep your content out of Perplexity entirely, you would need to block both, and since Perplexity says the user agent generally ignores robots.txt, that means a firewall rule. If your aim is simply to reduce crawling load, blocking user-requested fetches may frustrate people who are specifically trying to read your page through Perplexity. Decide based on your business goals and write the policy down.
The AI Visibility Tracker asks AI assistants the questions your customers ask and records whether your brand is cited, mentioned, recommended or missing, so you can confirm Perplexity is ignoring you before you start and see when that changes.
Open AEO Fixes โ