250 AI-Crawler Fetches, Zero Answers: The Pages We Built to Be Read, Not Quoted

Our blog category pages drew roughly 250 verified AI-crawler fetches each in 30 days and fed zero live answers. One dated changelog post at /post/google-ads-updates-2026 fed 435.

I found that split in our own edge logs. It overturned a rule I used to give clients: crawl equals interest, and interest eventually turns into citations. The bots reached the category pages. The pages just gave them little they could quote. That is the content gap worth finding: not always a missing page on the topic, but an existing page with no answer on it.

Setup: the pages that looked healthy in the logs

The category pages were built to be visited. New posts pointed back to them, internal links led to them, and their lists of excerpts changed as the blog grew. A crawler looking for updated content had good reason to return. When I first saw the fetch counts, I treated them as evidence that the pages were doing their job.

That was the wrong job to measure.

A verified fetch is a request whose user-agent and IP range match the AI operator’s published information. A user-agent alone is not enough; anyone can set one to GPTBot. A live-answer hit, as we counted it, records when that exact path supplied content to a live answer. The first number says a verified operator requested the path. The second says the path contributed to an answer. Those are not stages of one guaranteed process.

There is another distinction inside the first number. Training crawlers, search and answer crawlers, and on-demand fetchers visit for different reasons. A training fetch is not a citation opportunity in the same way an answer-related fetch may be. We were seeing substantial activity from answer-related crawlers on the category pages, then nothing from those paths in live answers. That was the first sign that access and usefulness had parted company.

The practical distinction is blunt: a fetch count tells you a page was requested, not that it answered anything.

First sign: 250 fetches, no usable passage

I opened the category page with the worst split and read it as an answer engine might. Title. Ten post excerpts. Pagination. The excerpts introduced their respective posts, but none finished the thought. To learn anything definite, a reader had to click through. That is acceptable navigation. It is a poor source for a standalone answer.

The page had the familiar signals of coverage. It named the topic, grouped relevant posts, and made them accessible. A conventional content-gap report would have counted it as a page on that subject. Yet the question facing an answer engine is narrower: what sentence here can stand on its own? AI engines quote passages, not just page titles. Ten introductions do not necessarily add up to one answer.

I should have stopped treating the category page’s repeat visits as a win. Instead, I let the fetch volume reassure me. Each new request looked like another chance for the page to appear in an answer, though nothing on the page had changed to make that more likely. More visits to the same list were not going to turn an excerpt into a claim.

That is the first wrong call in this postmortem: I read crawler activity as demand for the page’s answer when the page had no answer to give.

Wrong call: listing the subject instead of answering it

The category page was not the only place we made that choice. Its structure exposed the habit: write a soft introduction, point to something more useful elsewhere, and assume the page still counts as covering the question. It does for a site map. It does less for an engine trying to assemble an answer from a passage.

Pages can be skipped when the answer sits beneath introductory copy, the structure offers no clean passage to extract, or the text repeats a definition without an example or original data. Our list page managed a quieter version of the same problem. It did not bury one good answer. It split ten unfinished ones across ten excerpts. A crawler could find the page easily; an answer engine had little reason to quote it.

The missing ingredients were visible once I stopped looking at traffic. There was no direct answer near the top, no standalone claim with a number or date, and no short section that resolved one question before the next began. I am not arguing that every paragraph needs a statistic bolted to it. I am arguing that a page asking to be used as a source has to say something definite somewhere on the page.

A test across 10,000 queries by Princeton, Georgia Tech and IIT Delhi found that targeted edits lifted visibility in AI answers by up to 40%; quotations lifted it 37% and statistics 22%. Those figures do not promise the same result for our category pages. They do explain why I took the absence of a quotable passage seriously rather than calling 250 to zero bad luck.

The lesson at this point was not “publish ten more posts.” It was finish one useful thought on the page the crawler already visits.

Damage: even the proof page made the engine work too hard

Then I checked /our-results. That one stung more than a category page. It was meant to show a buyer what groas had achieved, yet it fed 1 live answer while /for-businesses fed 52. A navigation page failing to supply a quote is frustrating. A results page doing it suggests we have made our proof hard to use.

I read the two pages side by side. The results page said clients cut costs and scaled volume, but its claims depended on the adjacent videos for context. A person willing to watch could piece the story together. An answer engine looking for a line it could attribute had a harder task: who got which outcome, by how much, and over what period? The text did not carry those details together.

That is why a testimonial grid was not doing the work of a clear result sentence. The evidence may have been on the page, but the sentence an answer could lift was not. We had treated the layout as though its meaning would travel intact wherever the page was used. It did not even travel cleanly into our own text-only reading of it.

The damage was not a missing topic. We had a business page, a results page, and category pages. The paths existed. The gap was between having evidence on a page and stating that evidence in a passage. If a buyer has to watch a video to decode a claim, I should not expect an answer engine to reconstruct that claim on our behalf.

The contrast: one dated post gave each section a job

The changelog post at /post/google-ads-updates-2026 made the contrast hard to ignore. It fed 435 live answers in the same 30-day view. It carried a date in the URL and copy, used numbers throughout, and let each section answer one question before moving to the next. Unlike the category page, it did not merely point toward an answer elsewhere. Unlike the results page, it put the useful details in text.

That does not prove its date alone caused 435 answers. The useful mechanism is more ordinary: a reader or extractor can take one section and understand what it says without reconstructing the rest of the page. The date helps establish when the information applies; the numbers make the claims specific; the sections keep separate answers from running together.

Freshness also matters in the broader citation picture. One AI-visibility rundown reports that 83% of commercial citations come from pages updated within 12 months and 60% or more from pages updated within six months. It also reports that pages untouched for more than three months are three times more likely to lose citations. I would not use those figures as a schedule for changing every page. I would use them as a reason not to leave a useful claim floating without a date.

The comparison changed the question I asked before publishing. Not “does this page cover the topic?” but “can one section answer a question without borrowing meaning from the rest?”

Fix: rewrite the page the crawler already finds

I started with the category page that had the worst fetch-to-answer split. It already attracted crawls, so another page on the same subject was not my first move. I added 120 words at the top: what the category covers, a dated claim with a number, and a link to the post that supports it. That gave the page a self-contained block rather than another introduction to its list of links.

I do not count that edit as a citation win yet. The page has to be fetched again, and the live-answer number has to move. The edit addresses the defect I could see; it does not guarantee what an answer engine will do. That distinction is the whole reason this postmortem started with two numbers instead of one.

The results page needed a different version of the same repair. I turned claims that depended on the testimonial grid into plain sentences carrying the industry, outcome and timeframe together: a hospitality brand’s lead costs down 35% in 18 days; a cleaning business’s appointment volume up 52% at 31% lower CPA in 30 days. Those lines do not ask a reader to infer the result from a video beside them. The period belongs in the sentence, not in somebody’s memory of the layout.

I kept the bar simple: one passage must make sense when lifted out of the page. A direct answer near the top can do it. A short results sentence can do it. A section that answers one question before moving on can do it. A table or list helps when it presents information that is already clear; formatting cannot rescue ten claims that never quite get made.

There is a limit to the category-page fix. If the underlying posts are thin, a polished introduction to them is still a polished introduction to thin posts. Fix the posts first. In our case, the high verified fetch count made the existing category page the sensible place to test the rewrite. Rewrite where the crawler already arrives, then measure what happens.

The check I now run before calling a page a gap

This is where ordinary content-gap tooling can send a team in the wrong direction. A report that compares topics or keywords can tell you whether you have a page and a competitor has one you lack. It cannot, by that fact alone, tell you whether the page you have contains a usable answer. We would have passed a basic coverage check: category pages, a business page, a results page. We still had the 250-to-zero split.

I now separate topic coverage from quotability. Coverage asks whether the page exists. Quotability asks whether a passage on it states a useful claim with enough context to survive outside its layout. Neither is a substitute for checking whether an answer engine can reach the page in the first place.

For our 30-day check, that meant putting verified fetches by path beside live answers by path. It is the comparison built into groas that surfaced this case; our method for separating verified hits from noise explains the log side. I would not tell a team to commission another batch of pages from a coverage report until it had looked at the pages bots already request and the answers those pages actually feed.

To replay the check on another site, I would follow the same order:

1. Check access first. Pull 30 days of edge or server logs, look at requests from known AI agents such as GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot and CCBot, then verify user-agent and operator-published IP range rather than counting every claimed bot. Check status codes before editing copy. A 403 points to an access problem; a 200 shows the page was served, not that it was quoted. Cloudflare’s default blocking of AI crawlers on new domains since mid-2025 is one reason I check the firewall rather than guessing from robots.txt.
2. Put the two outcomes together. Export paths with verified fetch counts and join them to live-answer counts from an AI-visibility tracker. Separate high-fetch, zero-answer pages from low-fetch, zero-answer pages. The first group is where I look for an unusable passage; the second may need an access or internal-linking check before any rewrite. Do not let a zero denominator turn a sortable ratio into the entire diagnosis.
3. Read the high-fetch page without its layout doing the explaining. Can you point to a passage that states its claim and gives the context needed to understand it? If every excerpt trails off, every result needs an adjacent video, or every answer sits on another page, mark the passage as missing.
4. Change one section, then watch the next fetch and answer count. Write a direct two- or three-sentence answer where the page needs one. Keep a number, timeframe or date with the claim it explains. Recheck live answers after the next crawl cycle rather than declaring victory when the edit goes live.

That order matters. I have seen teams spend a month rewriting while a firewall rule blocked the bots they wanted to reach. Blocking search and fetcher bots can keep pages out of answers, while blocking training crawlers has different stakes. Copy cannot fix a 403. Equally, opening access cannot make a list of partial thoughts quotable.

The groas view puts those two sides next to each other because they call for different work. High verified fetches and no answers send me into the page. No successful fetches send me to access and discovery. Calling both problems a “content gap” without checking which one you have is how teams buy more articles and leave the useful fix untouched.

The rule that survived the postmortem

The category pages earned roughly 250 verified fetches each and fed zero live answers. The dated changelog fed 435. I cannot make every answer engine quote a rewritten page, and I will not count a fetch as a citation in advance. I can stop publishing pages that introduce ten other pages without answering a question themselves.

The single rule I apply now is this: before a page goes live, I must be able to lift one passage out of it and still understand the answer.