GEOFast
Sign inFree X-rayRun a free X-ray

PerplexityBot, the crawler behind Perplexity's search results

PerplexityBot is the crawler Perplexity uses to surface and link websites in its search results. Perplexity says it is not used to crawl content for AI foundation models, and recommends allowing it if you want your site to appear in those results.

Check your site for PerplexityBot

Checked against Perplexity's own documentation on October 13, 2026.

The crawler

What PerplexityBot is for.

Perplexity says PerplexityBot is designed to surface and link websites in search results on Perplexity, and is not used to crawl content for AI foundation models.[1]

  • Run by

    Perplexity

  • Its job

    Search index. It crawls on its own schedule, to find pages for search results.

  • Its name in robots.txt

    PerplexityBot

  • Does it follow robots.txt?

    Yes, by its operator's account. Perplexity gives PerplexityBot as a robots.txt tag for managing how a site interacts with it. Each setting works independently, and a change may take up to 24 hours to reach its systems.[1]

  • Its IP ranges

    Perplexity publishes PerplexityBot's IP addresses as a JSON file and calls it the source of truth for firewall rules, since the addresses are updated regularly.[1] perplexity.com/perplexitybot.json

  • Firewalls need a rule too

    Perplexity says a site behind a web application firewall may need to allow its bots explicitly, with a rule that matches both the user agent and its published IP ranges.[1]

User agent

The PerplexityBot user agent string.

Perplexity gives this as the full user agent string.[1]

  • Full string

    Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
robots.txt

The lines that allow and block PerplexityBot.

Each pair of lines is a group of its own in robots.txt. The robots.txt generator writes the whole file, and the robots.txt checker reads what yours says to PerplexityBot now.

  • Allow PerplexityBot

    With no robots.txt, or one that doesn't disallow it, PerplexityBot is allowed already. These lines say so by name: a crawler that finds a group with its own name follows that group, and follows the User-agent: * group only when it finds none.[2]

    User-agent: PerplexityBot
Allow: /

  • Block PerplexityBot

    Perplexity's page doesn't spell out what a block does. It says that to ensure your site appears in its search results, you allow PerplexityBot in robots.txt and permit requests from its published IP ranges.[1]

    User-agent: PerplexityBot
Disallow: /
Our data

How many cited sites block PerplexityBot.

We read the robots.txt of the 1,605 sites our recorded AI answers cited, for the 13 crawlers the X-ray reads for.

1.2%

of cited sites close their homepage to PerplexityBot in robots.txt: 19 of 1,529.

471

of 1,500 robots.txt files name PerplexityBot. 18 of those close the homepage to it and 453 leave it open.

Cited sites whose robots.txt closes the homepage to each crawler
CCBot4.2%
GPTBot3.1%
ClaudeBot2.6%
Google-Extended2.0%
ChatGPT-User1.3%
PerplexityBot1.2%
Perplexity-User1.0%
Claude-User1.0%
OAI-SearchBot0.9%
Claude-SearchBot0.8%
Applebot0.7%
Googlebot0.1%
Bingbot0.1%

Blue: PerplexityBot. Gray: the other 12 crawlers the X-ray reads for.

We took the 1,605 sites cited in 1,033 answers from ChatGPT, Claude, Perplexity, Gemini and Google's AI Overviews and AI Mode, recorded on October 6 and 7, 2026 for 54 questions about AI visibility tools and web analytics, and read each site's robots.txt as GEOFastBot with the X-ray's own rules on October 8, 2026. These are sites AI answers already cite, so sites that shut AI search crawlers out may be rarer here than on the web at large, and a site can serve other crawlers other rules than it served us. Shares are of the 1,529 of 1,605 sites whose robots.txt we could read, or that have none; 62 refused our request.
Same operator

Perplexity's other crawlers.

Perplexity gives each job its own name in robots.txt, so one can be allowed and another blocked.[1]

FAQ

Questions about PerplexityBot.

  • What is PerplexityBot?

    PerplexityBot is Perplexity's search crawler. Perplexity says it is designed to surface and link websites in search results on Perplexity, and is not used to crawl content for AI foundation models. Perplexity SEO

  • What is the PerplexityBot user agent string?

    Perplexity gives the full string as: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot).

  • Where does Perplexity publish PerplexityBot's IP ranges?

    At https://www.perplexity.com/perplexitybot.json. Perplexity says the addresses are updated regularly and recommends fetching the file on a schedule to keep firewall rules current.

  • Does PerplexityBot collect training data?

    Perplexity says it does not: PerplexityBot is not used to crawl content for AI foundation models. It surfaces and links websites in Perplexity's search results.

  • How many sites block PerplexityBot?

    In our study, 19 of 1,529 sites (1.2%) close their homepage to PerplexityBot in robots.txt. The base is the sites cited in AI answers about AI visibility tools and web analytics whose robots.txt we could read, or that have none, so other fields may differ. What the cited sites allow

Sources

Where each fact comes from.

Each numbered source is the page a fact comes from, most of them Perplexity's own documentation, read on October 13, 2026. The answers above repeat facts marked on the page. Vendors change these pages, so check them before you act.

  1. 1Perplexity: Perplexity Crawlersdocs.perplexity.ai/docs/resources/perplexity-crawlers
  2. 2IETF: RFC 9309: Robots Exclusion Protocolrfc-editor.org/rfc/rfc9309.html

Keep reading

Free X-ray

Find out what AI says about your site.

Paste your URL and leave with a ranked list of what to fix, with steps and code.

  • Free
  • No account