Two dated Google Ads changelog posts on our site drew 905 verified live-answer fetches in 30 days. Our /for-businesses page drew 68. If you are paying for AI search visibility, that gap points toward pages that answer a dated question, not another round of generic service copy.
It does not mean those posts appeared in 905 answers. A fetch tells us an assistant pulled a page while responding to a user; it does not tell us whether the answer cited it, named groas, or sent us a customer. That distinction is the point. I used to dismiss bot traffic reports because they counted everything. These edge logs let us separate verified live-answer fetches from background crawling and user agents merely claiming to be bots. Once you make that split, the pages worth studying look different from the pages with the loudest raw-hit totals.
The dataset: what we counted and what we cannot see
This is 30 measured days of edge-level logs from groas.com. Verified means the requesting IP matched the operator’s published list for that bot. Unverified means a request claimed a bot identity in its user agent but failed that check. Live-answer fetch means a user-triggered bot requested a page while answering a question, rather than a background crawler visiting it for later indexing.
Those labels are not interchangeable. A user agent is easy to claim; HUMAN Security found about 5.7% of requests claiming to be AI crawlers were spoofed. And when a CDN sits between a visitor and the site, the origin sees the CDN address. Verification needs edge-level signals, not an origin log interpreted as though it recorded the visitor’s IP.
Free tool: the AI Crawler Checker shows which AI crawlers your robots.txt lets in and what they receive when they arrive.
There is another limit: the path list available for this analysis is truncated. A URL missing from the visible rows did not necessarily receive zero fetches. I cannot rank every page on the site, or say these changelog posts were the sitewide winners. I can compare the visible paths with verified counts, show where raw traffic becomes misleading, and say what I would inspect before buying more content.
That is enough to make a decision. It is not enough to declare a universal citation rate.
Finding 1: dated changelogs drew the clearest content-page demand
The visible content-page contrast is sharp:
| Page | Verified live-answer fetches in the measured period |
|---|---|
| Dated Google Ads changelog post | 513 |
| Second dated Google Ads changelog post | 392 |
| Both posts combined | 905 |
/for-businesses | 68 |
The posts were about what changed in Google Ads and when. That gives an assistant a dated fact to retrieve when someone asks a dated question. A service page has a different job: it explains who we serve and what we do. I would not delete it because it drew fewer fetches. I would stop expecting it to answer a question about a recent change.
Outside this dataset, one analysis found that 76.4% of Perplexity’s top-cited pages had been updated within 30 days. That finding offers a plausible reason to take freshness seriously. It does not identify which assistant fetched each of our posts, or prove that freshness alone caused the difference. Topic, existing visibility, and the questions people asked also matter; our visible path counts do not separate those effects.
Still, the publishing implication is practical. If a buyer asks what changed this month, give the assistant a page that says what changed this month. Put the date and the change where a reader can find them. Make the answer easier to retrieve than the sales pitch.
Finding 2: live-answer bots do not all visit the same paths
The bot-and-path rows show why I would not roll every verified request into one grand “AI visibility” number:
| Bot and path | Verified live-answer fetches in the measured period | What the request shows |
|---|---|---|
ChatGPT-User, homepage / | 789 | A live request for the homepage |
Claude-User, /robots.txt | 890 | Repeated requests for access rules |
Claude-User, homepage / | 35 | Fewer live homepage requests than permission-file requests |
ChatGPT-User’s homepage requests are not evidence that all those users asked about groas by name. The logs show the page requested, not the wording of the prompt. Likewise, Claude’s requests for /robots.txt are not citations. They show that access rules featured heavily in the paths we could see for that bot. A permission-file fetch is not a content win.
The distinction also matters when you set access policy. ChatGPT-User is a user-triggered fetcher, while OAI-SearchBot has a separate search-crawling role. The same source says OpenAI’s robots.txt rules may not apply to ChatGPT-User requests. Anthropic’s Claude bots honor robots.txt. One rule of thumb for “AI bots” will not describe both behaviors.
I learned the danger of universal rules in paid search, usually after a neat account-wide assumption met a messy account. Here the check is simpler: inspect which bot requested which path, then inspect what your access rules allow. Do that before treating a homepage fetch or a robots.txt fetch as a recommendation.
Finding 3: raw media hits make a poor visibility report
The asset paths are where a raw-hit dashboard starts to flatter itself:
| Visible asset path group | Raw hits | Verified share |
|---|---|---|
| One image path | 2,395 | 282 verified |
| Icon paths | Not reported in the visible comparison | Under 5% verified |
I cannot label every unverified request a spoof. Some may be other automated traffic; the visible counts do not resolve each request’s purpose. What I can say is that raw hits on an image are not evidence that an assistant recommended the business. Even a verified asset request is not, by itself, an answer containing your brand.
A separate Cloudflare example recorded 897,000 AI requests, with a .webp blog image receiving 35,190 requests. That is a useful warning about media volume, not a benchmark for our site. It shows how easily an asset can dominate a request chart while telling you little about what a person saw in an answer.
There is a second narrowing step after retrieval: citation. In one measured set, about 110 of 3,554 retrieved pages appeared in the answer. Those are not our citation counts, and I would not apply that rate to our posts. The mechanism is the useful part: a system can fetch a page, inspect it, and leave it out of the final response.
So ask for the report in layers: raw requests, verified requests, live-answer fetches, and any separately measured citations or outcomes. If a vendor hands you the first layer and calls it demand, they are counting plumbing and calling it preference.
Finding 4: keep the commercial page, but give it the right job
The /for-businesses page received 68 verified live-answer fetches. That is less than either dated post in the visible rows, but it is not zero. A commercial page can answer a question about a company’s offer. A changelog answers a question about a change. They compete for different tasks, not for the right to exist.
I used to tell clients that one strong service page could carry most of this work. I was wrong to give it that much responsibility. It needs clear positioning that an assistant can understand. It cannot also be a current account of every rule, control, or cost that moved in the category. Stuffing a service page with updates would make both jobs harder.
An outside analysis reports that 87% of ChatGPT citations in its sample came from Bing top results, while its Perplexity findings emphasize more recent pages. That helps explain why one publishing approach may not serve every search surface equally. It does not let me assign our homepage fetches to ChatGPT prompts or our changelog fetches to Perplexity prompts. Our path table does not contain that answer.
The decision is narrower and more useful: keep the commercial page accurate and easy to read, then publish separate dated pages when there is a specific change worth answering. Do not make one URL pretend to be both a company introduction and a running change log.
What I would change in the content budget
I would fund pages that answer a dated, specific question before buying more undifferentiated service copy. The two visible changelog posts gave us a reason to test that priority; they did not give us a guaranteed fetch count for the next post. The writing brief is straightforward: identify the change, put its date in the title, and state the useful fact early. The draft advice to put it in the first 100 words is a good discipline, not a rule the logs can prove.
For a business outside paid search, the equivalent might be a change in its category, a cost that moved, or a rule that now affects buyers. Publish when there is something to say. A date on a page with no meaningful change is decoration, and an assistant does not owe it a fetch.
I would also change the reporting brief. Ask anyone selling AI search visibility for:
- Verified requests by page, with the bot identity checked against the relevant operator list rather than accepted from the user agent.
- Live-answer fetches separated from background crawling, so a content request is not buried beside routine indexing.
- Asset paths separated from content paths, so images and icons cannot inflate a page-performance claim.
- Citations and business outcomes reported separately, where they can actually be measured, rather than inferred from a fetch.
OpenAI publishes separate IP lists for its bots. Those lists can change, so a report based on a neglected hard-coded allowlist can drift. Behind a CDN, the verification needs to use edge data rather than assuming the origin recorded the requesting bot’s IP. If a vendor cannot explain that distinction, I would not let its raw-hit chart decide the next content retainer.
Check robots.txt before paying for another page
Claude-User requested our /robots.txt 890 times in the visible live-answer rows. That count is not a reason to optimize the permission file for “engagement.” It is a reason to make sure the pages you want read are accessible to the crawlers that honor it.
Open the file. Check whether it allows the blog and changelog paths you want available. Check for an accidental disallow on the pages you are paying to publish. Then check the policy for the bot whose visibility you care about: OAI-SearchBot and ChatGPT-User do different work, so a ChatGPT-User fetch alone does not establish that the search crawler can reach the page. The practical order is access first, content second, measurement third. Paying for writing while blocking the intended reader is expensive comedy.
The same caution applies to the report you get back. On Cloudflare Free, crawler identification described in this monitoring example relies on user-agent strings, while paid plans offer additional bot-detection signals. A claimed bot name in a report is not the verification standard used in this analysis. Ask which signal the report used before you treat a request as a confirmed visit.
This is the work I would expect from a company managing AI search visibility on an ongoing basis: keep the right pages reachable, publish dated answers when the subject warrants them, and show verified live-answer requests by page without pretending those requests are citations. groas offers SEO and AI Search work alongside technical fixes and new pages, with attention to downstream revenue rather than raw bot counts. The report should make the difference between those measures plain.
Thirty days on one site cannot tell us which prompts lead to citations across ChatGPT, Perplexity, and Gemini in general. A truncated path list cannot establish a sitewide winner. But these logs do support a spending decision: inspect access, separate verified live-answer fetches from noise, and give dated questions their own pages. Run that split on your domain before you approve another content retainer.

