How ChatGPT, Perplexity and Google AI Choose Which Sources to Cite

The pipeline in plain English#

A language model on its own only knows what it absorbed in training. To answer with current facts and links, AI search adds a retrieval step. The industry name is retrieval-augmented generation (RAG). Google calls it grounding: its guide says the systems rely on core Search ranking systems to retrieve relevant, up-to-date pages from the Search index, then link to supporting sources.

1. 1

   1. The user asks

   A prompt arrives, often long and full of context: “Which invoicing tool works for a five-person design agency that bills in USD and EUR?”

2. 2

   2. Query fan-out

   The model breaks the prompt into several related searches. Google defines fan-out as concurrent, related queries generated by the model to request more information. OpenAI says ChatGPT search typically rewrites your query into one or more targeted queries and may send follow-ups after reviewing results.

3. 3

   3. Retrieval from an index

   Each sub-query hits a search index. Google's AI features use the Google Search index. ChatGPT search uses OpenAI's OAI-SearchBot crawl plus third-party search providers (Microsoft announced Bing as ChatGPT's default search in May 2023; OpenAI's help page does not name current partners). Perplexity uses PerplexityBot. Claude's search crawler is Claude-SearchBot; Anthropic does not document which index Claude searches, though its subprocessor list has included Brave Search since March 2025.

4. 4

   4. Passage selection

   The system reads the retrieved pages and keeps the parts that answer each sub-query. A clear paragraph that directly answers one sub-question is easier to use than a strong page where the answer is buried.

5. 5

   5. Synthesis

   The model writes one answer from the selected material plus what it already knows. Facts that several sources agree on are easier to state with confidence.

6. 6

   6. Citations

   Some of the sources used get linked. Not every source used is cited, and the set changes between runs.

Who crawls for what#

Two details trip people up. First, OpenAI says a site should allow OAI-SearchBot to be included in ChatGPT search; blocking only GPTBot opts you out of training, not search. Second, Google says Google-Extended does not affect inclusion in Google Search and is not a ranking signal. User-triggered agents such as ChatGPT-User and Perplexity-User may not follow robots.txt, because a person asked for that page.

Two ways to be known#

• Training data. The model learned about your brand from the web before its cutoff. This shapes what it says even with search off, and it changes only when new models ship. Training crawlers (GPTBot, ClaudeBot) feed this path.
• Live retrieval. The engine searches at answer time and reads your current pages. This is where citations come from, and you can influence it this week. Search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot) feed this path.

Most AEO wins come from live retrieval because it responds to changes you make now. Training-data presence matters for brand-level questions (“What is LedgerLoop?”), and it is built the slow way: being widely and consistently described across the web.

What this means for your content#

1. Be retrievable. The right crawlers are allowed, pages are indexed, and the main content is in the HTML. Vercel's December 2024 analysis found that none of the major AI crawlers it studied rendered JavaScript, apart from Google's and Apple's. Server-render anything you want quoted.
2. Be passage-level relevant. Each important question gets its own clearly headed section that answers it in the first sentence or two. Fan-out means your page competes sub-question by sub-question.
3. Be corroborated. Engines combine sources. A claim only you make is weaker than one that reviews, directories, forums and publications repeat. Off-site mentions are part of the job.