The four big engines do not share one method, but they follow the same arc. The model decides whether the question needs the web, turns it into search queries, reads what comes back, writes an answer and attaches the sources it used. Each step is a place where your brand can drop out. This guide walks through the arc using what OpenAI, Anthropic and Google say in their own documentation, and it marks where the documentation stops and guesswork starts. You will not find percentages here, because none of the vendors publish numbers that would back them.
Memory or search
An answer is built from the model’s memory, from a web search, or from both. A memory-only answer has no sources, so it cannot cite you. It can still mention you, if you appeared often enough in the training data. Whether an engine searches is the engine’s decision.
- ChatGPT. OpenAI’s help center says ChatGPT may search the web automatically when a question would benefit from current information, and that people can switch Search on by hand.
- Claude. Anthropic’s documentation says Claude determines when to search based on the prompt. It searches when a request depends on information that is current, changing or outside its training data, including recent events, current prices and facts about specific organizations or products that might have changed. It answers directly for stable knowledge, such as established facts or conversational turns.
- Gemini. Google’s Gemini API documentation describes grounding with Google Search: the model analyzes the prompt, decides whether a search can improve the answer and, if so, generates one or several search queries and uses the results.
- Perplexity. Perplexity is built around search. Its crawler documentation says PerplexityBot is designed to surface and link websites in search results on Perplexity.
For a brand, the practical point is that questions about current facts, such as “pricing”, “best tool for” and “alternatives to”, are the ones most likely to trigger a search. Those are also the questions buyers ask.
From one question to several queries
Engines rarely search the buyer’s exact words. OpenAI’s help center says ChatGPT search typically rewrites a question into one or more targeted queries and, after reviewing the first results, may send further, more specific ones. Its own example turns a question about the latest drug development into a query that adds the year. Google says AI Overviews and AI Mode may use a “query fan-out” technique, issuing multiple related searches across subtopics and data sources.
The consequence is that you are not competing for the buyer’s sentence. You are competing for the sub-questions behind it: how much it costs, who the alternatives are, how it compares, what it integrates with, who it suits. A site with one page that tries to answer all of that gives the engine less to cite than a site with a clear page for each.
Where the results come from
Search results come from an index, and being absent from the index ends the story before ranking starts.
- OpenAI. ChatGPT search uses OpenAI’s own crawler, OAI-SearchBot, and sometimes partners with other search providers. OpenAI says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links. To be eligible, allow OAI-SearchBot and OpenAI’s published IP ranges. Inclusion is not guaranteed.
- Google. AI Overviews, AI Mode and Gemini’s Google Search grounding draw on Google Search. Google says a page must be indexed and eligible to be shown in Search with a snippet to appear as a supporting link, and that there are no additional technical requirements.
- Perplexity. PerplexityBot builds the index behind its search results. Perplexity says changes to robots.txt can take up to 24 hours to take effect.
- Anthropic. Claude’s web search tool runs searches through the API and returns the results to the model. Anthropic documents separate crawlers for training (ClaudeBot), for fetches at a user’s request (Claude-User) and for search quality (Claude-SearchBot).
You can see how your own robots.txt treats these crawlers with the AI crawler check.
What gets cited from the results
This is the part no vendor documents. There is no public rule for why one retrieved page is cited and another is not. What is observable and sensible is short:
- Relevance to the rewritten query. A page that answers the narrow question beats a page that mentions it in passing.
- Easy extraction. The answer is stated plainly near the top, under a heading that matches the question, or in a list or table.
- Freshness for facts that change. Query rewriting adds dates, and search results can carry a page age; Claude’s web search results include one. Keep pages whose facts change, such as pricing and features, visibly current.
- Third-party pages. In practice you will see review sites, roundups and documentation cited next to, or instead of, vendor pages. Being described well on those pages is part of the job.
How citations show up
A citation is a link to a specific URL, not to a brand. That means which of your pages gets cited matters as much as whether you are cited.
- OpenAI’s API returns a url_citation annotation with the URL, title and location of each cited source, and ChatGPT shows inline citations and a Sources list when available.
- Anthropic’s API always enables citations for web search, and each one carries the URL, the page title and up to 150 characters of cited text.
- Gemini’s grounded responses carry inline annotations with the citations.
- Perplexity attaches linked sources to its answers.
OpenAI’s own help page adds a warning worth repeating: search results and citations can be incomplete, outdated or incorrect, so open a cited source to check that it supports the answer.
What to do with this
- Allow the search crawlers for each engine and confirm that your CDN or firewall does not block them.
- Write one clear page per buyer sub-question: pricing, alternatives, comparisons, integrations, use cases.
- Keep changing facts current and visibly dated.
- Look at what is cited today. The citations in a scan report show which pages the engines already trust for your category, and where you are missing.
Engines change, and so do their documents. Treat this as a description of how things work today, and verify it by measuring. Our measurement guide shows how.