Our AI Bot Logs Had 14,742 robots.txt Hits. Only 3,894 Were Verified.

Only 3,894 of 14,742 requests to our robots.txt file that claimed to be AI crawlers passed an IP verification check. If an agency had reported the raw count as AI attention, roughly three quarters of its headline number would have been unverified.

We logged 30 days of requests at the edge on groas.com. The more useful finding came after verification: two timely Google Ads changelog posts drew 922 live-answer fetches between them, while a teardown post with 309 claimed bot hits drew just one. Those are fetches, not confirmed citations or answers that displayed our pages. But they make the reporting decision fairly clear. If I were choosing an AEO tool for an agency, I would ask for verified live-answer fetches per client page, not another prompt-rank chart.

What our edge logs can—and cannot—count

I used to tell clients that bot traffic meant interest. I was wrong. A user-agent string is a claim made by the requester; it is not proof of identity. For this analysis, we logged requests on groas.com whose user agents claimed to be AI bots, then checked their source IPs against the operators’ published ranges. That distinction between a claimed bot and a verified one is the verification rule behind every table here.

The source for our site figures is our own 30-day edge-log window. These are request counts, not analytics sessions, people, prompts or citations. The IP check tells us whether a request came from a published operator range. It does not tell us what a person asked, whether the fetched page appeared in an answer, or how often that answer was seen.

We then separated verified requests by job. Operators use different bot classes: training crawlers such as GPTBot and ClaudeBot, search or index crawlers such as OAI-SearchBot and PerplexityBot, and user-triggered fetchers such as ChatGPT-User and Claude-User. A request from the last group is the closest log-level signal here that a page was pulled for a live answer. It is still a fetch, not proof that the answer cited the page.

That is the order to read the figures below: identity first, purpose second, citation separately. Server logs can show a fetch, and a 200 response means the page was available to read; they cannot establish a citation on their own (what server logs prove).

Finding 1: the raw bot count shrinks under verification

The largest claimed count appeared on robots.txt: 14,742 requests in 30 days. Of those, 3,894 verified, a 26.4% share. Put another way, 73.6% of the requests claiming an AI-crawler identity did not pass our check. Image requests were less convincing still: 87 of 1,271 verified, or 6.8%.

Source: groas.com edge logs, one 30-day window. “Verified” means the source IP passed our check against published operator ranges; a request that failed the check is not, by itself, proof of who sent it.

The mechanism is less mysterious than the dashboard makes it look. Anyone can send a request bearing ChatGPT-User as its user agent. Checking the source IP changes the question from “What did this request call itself?” to “Did it come from the operator it named?” An independent log analysis found a similar distinction in a smaller sample: six of 33 claimed live fetches verified, while some unverified requests sought credential files rather than page content. Those were not our requests. The comparison illustrates why a bot label alone is weak evidence, not why any particular request in our logs failed.

Practical takeaway: if a report leads with raw AI-bot hits, ask how many passed an IP check before calling the number attention.

Finding 2: live fetches favour a few pages

Verification clears up who made a request. The fetcher class helps explain what kind of request it was. On groas.com, verified live-answer fetches concentrated on the homepage and two timely changelog posts. The teardown post shows why claimed traffic is a poor shortcut to that conclusion.

Source: groas.com edge logs, the same 30-day window. Approximate counts are marked with ~. Live-answer fetches are verified requests from user-triggered fetchers, not a count of published answers or citations.

The two changelog posts drew 561 and 361 live fetches, respectively. Against their claimed hit totals, that is roughly 65% and 75%. One of them also had 845 verified requests out of 868 claimed, a sharp contrast with the robots.txt and image rows. The posts were timely answers to questions about Google Ads changes. The logs show that live fetchers repeatedly requested them; they do not show the prompts that led there or guarantee that the pages were cited.

Now read the teardown row from left to right. Its 309 claimed hits could have made it look busy in a raw-traffic report. Six passed verification. One was a live-answer fetch. If I had sorted the page report by claimed hits and stopped there, I would have given that post far more credit than these logs support.

Practical takeaway: put verified live-answer fetches beside each client URL. A sitewide bot total hides the page that was pulled for a live answer—and the page that merely attracted requests wearing a bot name.

Finding 3: the fetcher matters as much as the page

Of the homepage’s 795 live-answer fetches, 751 came from ChatGPT-User. Claude-User appeared in a thinner pattern in this window: fewer pages and fewer repeats, with more activity on the homepage and the newer changelog post. That is a useful split for our reporting, not a ranking of which engine matters most to every client.

It also explains why I would not roll every verified AI request into one “AI visits” line. A training crawler, search crawler and user-triggered fetcher do different jobs. A verified training crawl says something different from a verified request made while a tool is assembling an answer. Both may matter to a site owner, but adding them together erases the distinction this analysis is trying to find.

Control is another reason to keep the classes separate. User-triggered fetchers may treat robots.txt differently from crawlers doing routine collection, and robots.txt is not access control. If the decision is to block access, a robots directive should not be mistaken for enforcement; that calls for access rules such as WAF rules. Declared identity has limits, too: Cloudflare’s account of de-listing Perplexity as verified describes both declared requests and an undeclared crawler rotating IPs past blocks. That account is context, not a measurement from groas.com.

Practical takeaway: report the page and the verified fetcher together. “AI bots visited” is too broad to tell a client whether live-answer systems pulled the work they paid to publish.

Why a prompt-rank chart is not the answer

A prompt tracker can say a client ranked third for a set of prompts. It cannot, by itself, show whether a verified fetcher pulled the client’s page during the period being reported. Nor is one prompt run a stable view of citations. In repeated ChatGPT runs of the same prompt, a median 7.7% of cited domains appeared in every run, with 0.54 overlap between consecutive runs. That is a reason to show repeated-run citation frequency rather than present one screenshot as a settled position.

Clicks are a separate limit on what a rank chart can promise. A roundup of AI-search reporting data cites Seer’s organic click-through change from 1.41% to 0.64% where AI Overviews appeared, and Pew’s 8% click rate with an AI summary versus 15% without one. Those figures do not measure our pages or prove the effect of any client’s citation. They do make it harder to treat placement in an answer as interchangeable with a visit.

I stopped sending rank screenshots as the main event. The more defensible report has two views side by side: verified live-answer fetches by client page and fetcher, then citation frequency across repeated prompt runs. The first says which pages the logs show being pulled for live answers. The second checks how often monitored prompts produced a citation. Neither substitutes for the other, and neither is closed-won revenue.

Our teardown post is the useful test case. A raw count gave it 309 claimed hits. Verification reduced that to six, and the live-fetch cut reduced it to one. A prompt dashboard on its own could not have shown that progression. Before paying for a chart, I want to know whether the tool can make those cuts.

The agency-tool trial checklist

For an agency juggling multiple client accounts, a solo-brand prompt tracker can leave the actual reporting work in spreadsheets. An agency-tool comparison lists multi-brand workspaces, white-label reporting, bulk prompt management, API access, per-brand pricing and competitor share of voice among the buying considerations. It also lists Otterly AI from $29 a month, Peec AI from $89, Profound Growth at $399 and Scrunch Agency Core at $500. Those are listed starting prices, not evidence that any of those tools passes the log checks below. Use a trial to test the output, not a sales call to imagine it.

1. Claimed versus IP-verified hits for each client domain. Ask to see the verification rule; a user-agent count alone can include unverified requests.
2. Verified live-answer fetches per URL and fetcher. Separate ChatGPT-User, Claude-User and comparable user-triggered fetchers from training and indexing crawlers.
3. A page table that keeps claimed, verified and live counts together. A live rate without its counts can conceal a page with almost no activity.
4. Engine split and competitor share of voice. Keep comparisons tied to named competitors and monitored prompts.
5. Month-over-month movement beside named fixes. Do not make the client infer a cause from a line moving on a chart.
6. White-label export and bulk prompt control across client brands. Otherwise the team assembles the report by hand.
7. Repeated-run citations beside the fetch log. Skip this pairing and you can mistake a fetched page for a cited one, or a single prompt result for a durable position.

What this month does not prove

This is one domain over 30 days. Groas.com publishes in a niche where timely posts about Google Ads changes can match questions people are asking now. That helps explain why the changelog posts stood out, but the logs do not isolate freshness as the cause. Their roughly 65% and 75% live-fetch shares are not a forecast for an evergreen services page, another client, or next month.

The logs also do not prove that any of the 922 fetches for those two posts became a visible citation. A successful page response shows availability to read. An answer may still draw on other sources or cite none. This is why I would pair logs with repeated prompt checks rather than rename a fetch count “answers fed” and hope nobody asks what it means.

Finally, ChatGPT-User dominated the live-fetch pattern on our homepage in this window. A local services client might have a different engine split. The method travels; these shares do not. Verify identity, separate the fetcher’s job, and compare pages before drawing a client-level conclusion.

The buying decision is plain. Stop paying for reports that treat claimed bot hits as attention or a single prompt rank as a result. Ask for verified live-answer fetches per client page, split by fetcher, with repeated-run citation frequency alongside them. That is the strongest read-to-citation view these numbers support without pretending a fetch is a citation.

For agencies, groas pairs paid and organic search execution with branded weekly reports under your name. Put any AEO reporting tool, including ours, through the same test: can it show what was claimed, what verified, what was pulled for a live answer, and what appeared in repeated citation checks? If it cannot show the fetch first, keep your money.