PerplexityBot

PerplexityBot is Perplexity's crawler, responsible for indexing pages into the source pool its answer engine retrieves from and cites, functioning as the crawler equivalent of Perplexity's real-time, citation-first retrieval approach. Unlike some competitors, Perplexity runs one primary crawler rather than separating training and citation functions.

Why it matters

Because Perplexity operates a single crawler for both discovery and citation, blocking PerplexityBot removes a page from Perplexity entirely, with no equivalent of OpenAI's or Anthropic's split between a training-only bot and a citation-specific one.

Related terms

• Perplexity

  Perplexity is an AI-native search engine built around inline, numbered citations, where nearly every claim in its answers links directly to a specific source. It positions itself explicitly as a citation-first answer engine, and its retrieval favors recently published, directly responsive pages over pages with heavier accumulated editorial authority.

• Real-Time Retrieval

  Real-time retrieval is Perplexity's defining retrieval strategy: prioritizing recently published, directly responsive sources over pages with heavier accumulated editorial authority. Instead of leaning on a domain's long-term reputation, Perplexity's index favors whichever page answers the immediate query most precisely, updated continuously as new pages get published or existing ones get revised.

• robots.txt (AI directives)

  robots.txt is the long-standing standard crawler-control file at a site's root, now extended with AI-specific user-agent rules that separately allow or disallow crawlers like GPTBot, ClaudeBot, and PerplexityBot. A single robots.txt file can permit standard search crawling while blocking one or more AI crawlers, or the reverse.