A page can answer a buyer’s question perfectly and still never appear in a ChatGPT or Perplexity citation. If the engine cannot find, fetch or extract that answer, the quality of the prose is beside the point.

I spent years running search accounts for founders who wanted to know why a good page was not getting seen. The answer was rarely “write more authoritatively.” Distribution comes first. Getting cited is a retrieval problem before it is a content problem.

Follow one buyer question through the system: what is the average cost of commercial air compressor preventive maintenance contracts for machine shops? The exact route varies by engine and query, but the useful model has five gates: the question becomes searches; those searches produce candidate pages; the engine tries to fetch them; it selects passages that answer; and other available evidence can affect which answer it trusts. A page that misses an early gate never gets a fair hearing at the later ones.

1. Your buyer’s question becomes several searches

A buyer asks about the average cost of a maintenance contract. That sounds like one topic. To retrieve an answer, an engine may break it into narrower searches: contract pricing, compressor type, service frequency and the kind of facility involved. This is query fan-out.

An incoming buyer question splits into several narrower search queries.

For our example, plausible searches include industrial rotary screw compressor service contract pricing and machine shop compressed air maintenance cost benchmarks. These are illustrations of the mechanism, not a transcript of what an engine searched. Dan Petrovic’s examination of ChatGPT retrieval offers a closer look at how the system works.

Here is the first failure mode: your page answers the broad question in its title but never names the component the search is looking for. A page headed “Everything You Need to Know About Compressed Air” might contain three screens of manufacturing background before it reaches maintenance contracts. A service page that names the equipment and describes the contract may be a better candidate for the narrower search, even if the first page is more polished.

This changes what a content gap means. It is not just a missing article about a buyer’s exact prompt. It may be a missing answer to one of the smaller questions inside it. If the search never surfaces your URL, better prose farther down the page cannot win the citation. Start by asking which concrete parts of the buyer’s question your page actually covers.

2. The search index decides which pages get a chance

Those narrower searches still need somewhere to run. Answer engines draw candidate URLs from search indexes; they do not generate a list of source pages from memory alone. That is why ordinary search visibility still matters, even when the final answer does not resemble a Google results page.

Indexed pages narrow into candidate documents and answer passages.

The distinction is which search you measure. In an Ahrefs study of 15,000 queries, 12% of URLs cited by ChatGPT appeared in Google’s organic top 10 for the exact prompt query; the reported overlap for Perplexity was 28.6%. That does not prove rankings are irrelevant. The buyer’s wording is not necessarily the wording used to retrieve a source. A page may be visible for a narrower equipment or pricing query without ranking for the whole conversational prompt.

There is also a difference between finding a page about a topic and finding the part of that page that answers it. As Perplexity engineer Johnny Dwyer discussed in an interview with Search Engine Journal, sub-document indexing lets a system work with portions of pages rather than treating each document as one indivisible result. For the air-compressor question, a maintenance-cost section can matter more than the general subject of the homepage.

Older pages appeared prominently in Ahrefs’ analysis of 1.4 million prompts: cited pages had a median age of roughly 500 days. I would not turn that observation into a rule that a new page cannot be cited. It is a reminder that publishing a page and making it retrievable are different jobs. Check whether the relevant page is indexed and visible for the narrower searches before rewriting its introduction.

3. A candidate page still has to be readable on fetch

A URL in the candidate set is not yet a usable source. The engine may need to fetch its content, and the request can fail before any answer is evaluated. This is where a blanket bot rule, an edge firewall or a JavaScript-only page can turn a perfectly relevant URL into an empty promise.

First, distinguish crawlers by job. OpenAI’s bot documentation distinguishes GPTBot, used for training-related crawling, from OAI-SearchBot, which serves ChatGPT Search indexing, and ChatGPT-User, which can fetch a page in response to a user’s request. A decision about training access is not automatically a decision about search access. If you block OAI-SearchBot, do not assume your page remains available for ChatGPT Search indexing.

Perplexity likewise distinguishes PerplexityBot, which builds its index, from Perplexity-User, which handles user-initiated requests. Its documentation says Perplexity-User generally does not follow standard robots.txt disallows. An edge firewall is a separate obstacle: it can return a 403 Forbidden or 429 Too Many Requests response to a request that otherwise could have reached the page.

Then inspect what a successful request receives. If the maintenance-contract table exists only after client-side JavaScript runs, the initial HTML may contain little more than a container. A browser user sees the prices or service details; a fetcher reading that initial response may not. In our teardown of AI crawler logs, technical fetch failures were a bigger issue than weak writing. The practical lesson does not require guessing an engine’s exact time limit: put the answer in HTML a fetcher can read.

An empty initial HTML shell beside a rendered technical service table.

Standard analytics will not necessarily show a visit stopped at the firewall. Server-log guidance for AI bots is useful here because the request and its status code tell a different story from a traffic dashboard. Before editing a passage, confirm that the relevant bots can reach the URL and receive the answer-bearing HTML.

4. The engine needs a passage it can actually quote

Fetching a page is not the same as selecting it. An answer engine needs a compact passage that addresses the question without forcing it to reconstruct the answer from scattered hints. Perplexity describes query-aware context compression: selecting useful source material while discarding text that does not help answer the query.

Return to the maintenance page. “We provide reliable agreements designed to maximize uptime” says little about contract cost or what a contract includes. A passage that identifies the compressor class, states which services the agreement covers, and gives a price range if the business has one it can support gives the engine something to use. Do not invent a number to fill the gap. If pricing depends on a quote, say what determines the quote and which services are included. That is more useful than a claim about being “industry-leading.”

The position and shape of that answer matter too. A section headed “Maintenance contract scope and pricing” is easier to find and interpret than the same information buried under “Things to Consider.” Define the subject close to the details; do not make a passage depend on a pronoun whose referent sits four paragraphs earlier. Write the part a reader would want quoted as a self-contained answer. You can still write a good article around it. The machine just needs a clear place to start.

5. A clean answer is not the only evidence available

Suppose two pages provide readable, relevant maintenance-contract details. The engine can also encounter third-party material about the same equipment, services or provider. That outside context may strengthen one candidate and cast doubt on another. It is not a guaranteed final “corroboration step” applied identically to every answer, and it cannot be diagnosed from a server response code.

The useful principle is narrower: a claim that exists only on your own page has less independent context around it. If the maintenance information also appears in relevant industry discussions or other third-party sources, an engine has more to compare it with. If your page makes a striking claim that nothing else supports, clean formatting alone will not make it persuasive. Get the underlying claim right before worrying about how neatly it can be extracted.

Test the first gate you might be missing

I would test retrieval before commissioning a rewrite. The question is simple: does the engine find and successfully fetch the page you expect it to cite? The setup should help you separate that observation from your theory about why a citation did or did not appear. If you want a fuller example of the approach, see our 30-day server-log test for AI citations.

Set up the test prospectively:

  1. Choose 10 buyer prompts. Use multi-part questions your prospects might ask, including the equipment, setting and decision they care about. Record the page you expect to answer each one.
  2. Prepare the logs. Export server or CDN access logs with timestamps, request paths, status codes and full User-Agent strings. Make sure the view you use includes requests stopped at the edge; origin logs alone may miss them.
  3. Set the controls. Run the same saved prompts on each engine, note when you ran them, and avoid changing the target pages during the test. Record citations as they appear rather than relying on memory.
  4. Build the sheet. Track the prompt, likely narrower searches, target URL, relevant bot requests, returned status and citation result. Mark a narrower search as your hypothesis unless you can actually observe it.

Run the prompts through ChatGPT Search and Perplexity, then inspect requests near each run. A quick filter for live user agents in an Nginx-style log is:

grep -iE 'ChatGPT-User|Perplexity-User' /var/log/nginx/access.log | awk '{print $7, $9}' | sort | uniq -c

That command is a starting point, not a complete attribution method. It collapses request paths and status codes across the file; timestamps and your CDN records help you connect an event to a particular run. Keep index crawlers separate from live user agents. Count fetch attempts for the pages you expected to win, inspect their status codes, and check the HTML returned by successful requests.

My expectation is that this will expose an earlier failure for some pages that looked like writing problems. Interpret the outcomes in order:

  • No observed request: Check indexation and visibility for the narrower searches. A missing hit in the logs you have does not, by itself, prove the page was never considered; the engine may have relied on indexed material or a request your logs do not show.
  • A 403 or 429: Investigate the edge rule or rate limit that returned it. The request reached a barrier, not an answer. Confirm what happened before assuming one rule change will solve every citation problem.
  • A 200 with an unreadable initial response: Fix delivery. Put the relevant contract details in the HTML the fetcher receives.
  • A readable 200 without a citation: Now inspect the passage and the other sources the answer used. A successful fetch proves delivery, not extraction, trust or selection.

A result either way changes the next task. An access failure sends you to the firewall or rendering setup. A readable page that loses to another source sends you back to the actual answer and its supporting context. That is a better use of 40 hours than rewriting every service page because a visibility chart went down.

One more case: the fast page that still loses

Imagine a regional distributor with a static, easily fetched service page. Its maintenance-contract section names rotary screw compressors and lists the services included, but says nothing about how pricing is determined. A larger manufacturer publishes a slower, more elaborate page with a readable section on contract pricing. The buyer asks our original cost question.

Work through the gates. The distributor might enter the candidate set for searches about compressor maintenance. Its page might return a clean 200, and the service list might extract beautifully. None of that makes it an answer to average cost. The manufacturer’s pricing section is a better fit for that part of the question if it can be retrieved and read. The distributor does not need more introductory prose, and its fast server is not the missing ingredient. It needs to decide whether it can publish useful, supportable pricing information. If it cannot, it should not invent a range for a citation.

That is the point of the model: find the first gate the page fails, then change the thing responsible. A missed citation is an outcome, not a diagnosis. At groas, the work starts with that distinction: autonomous execution across paid and organic search, with a human strategist setting direction and owning accountability, rather than handing you another dashboard of missing mentions. Whether you work with us or run the test yourself, fix the earliest broken step first. A better answer matters once the engine can find and read it.