---
title: "15,467 Bot Fetches. Still No Proof of an AI Recommendation."
description: "groas logs show 15,467 claimed bot fetches, 3,939 verified fetches and 890 live-answer retrievals. Here’s what to measure before paying for AI visibility."
url: "https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/e72741c3-5b6c-4ff7-8084-21608cd55913.png"
published: "2026-11-19T11:02:00.578Z"
modified: "2026-11-19T11:02:00.657Z"
---

November 19, 2026 · 9 min read

# 15,467 Bot Fetches. Still No Proof of an AI Recommendation.

[](https://groas.com/author/alexander-perelman)

[Alexander Perelman](https://groas.com/author/alexander-perelman)Personally Tested

Head Of Product @ groas

[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![Tiny model shop where hundreds of figurines crowd a rules sign at the gate beside a giant tally counter, while the shopkeeper inside waits alone at an empty counter.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/e72741c3-5b6c-4ff7-8084-21608cd55913.png)

In this article

1. [The biggest count belongs to robots.txt](#the-biggest-count-belongs-to-robotstxt)
2. [The smaller article has the stronger live-answer signal](#the-smaller-article-has-the-stronger-live-answer-signal)
3. [Retrieval, citation and recommendation need separate evidence](#retrieval-citation-and-recommendation-need-separate-evidence)
4. [Build a baseline before buying another dashboard](#build-a-baseline-before-buying-another-dashboard)
5. [Pay for changes the measurement can inspect](#pay-for-changes-the-measurement-can-inspect)

Of [15,467 claimed AI-bot fetches of one groas infrastructure file, only 890 were classified as live-answer retrievals](https://groas.com/post/we-logged-every-ai-bot-hit-on-our-site-f). Neither number proves a single AI recommendation.

That is the problem with cheap AI-visibility reporting: the biggest number can measure the thing furthest from a buyer choosing you. In the supplied 30-day measurement, verification reduced 15,467 user-agent-matched fetches to 3,939. Separating live-answer activity from background crawling reduced the count again.

Each number answers a different question. Did a request claim an AI-bot identity? Did its source pass verification? Was it associated with a live answer session? None answers the question a budget-conscious buyer actually cares about: did the answer recommend the brand?

I keep those questions separate because **access is not endorsement**. A fetch makes a page available to the system. A citation credits a source. A recommendation tells the buyer what to choose. Counting the first and selling it as the last is how cheap reporting gets expensive.

## The biggest count belongs to `robots.txt`

The file behind that headline number is `robots.txt`. It tells crawlers where they may go; it does not answer a buyer’s question. High activity here is evidence of infrastructure requests, not demand.

Here is the supplied 30-day breakdown for that file:

| Measurement                | What it records                                                                      |  Total |
| -------------------------- | ------------------------------------------------------------------------------------ | -----: |
| User-agent-matched fetches | Requests claiming an AI-bot name                                                     | 15,467 |
| Verified fetches           | Requests whose source passed the reported verification check                         |  3,939 |
| Live-answer fetches        | Verified requests classified as live-answer activity rather than background crawling |    890 |

A user agent is a claim, not proof. Any scraper can write `GPTBot` in that field. A [spoofed crawler](https://angeo.dev/spoofed-ai-crawler-definition/) is a non-vendor client borrowing that identity.

Verification requires more: matching the source against a vendor’s published IP ranges or checking a valid signature where available. The method matters. [GPTBot can be checked against OpenAI’s published ranges, while ClaudeBot publishes no IP ranges](https://www.aaivisibility.com/blog/verify-gptbot-ai-crawler-ip-check). A report needs to explain which identities it can verify and which remain merely claimed.

That distinction also limits the conclusion. **Traffic that did not verify is not automatically proven spoofed.** It belongs outside the verified count unless there is evidence to classify it more specifically.

The verified bucket then needs another split. [Background crawls and user-triggered fetches measure different activity](https://revnu.com/blog/ai-crawler-tracking-without-javascript-what-logs-show). Combining them produces a larger number, but not a clearer account of whether a buyer’s answer was involved.

My rule: ask how each bucket was defined before paying attention to its size.

![Bar chart comparing claimed AI-bot fetches, verified fetches and live-answer retrievals](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/5c98cb0f-5702-4d6e-8c85-7e2c3bfbbb4a.png)

## The smaller article has the stronger live-answer signal

Put the infrastructure file beside the homepage and a Google Ads updates article, and the weakness of a single total becomes obvious. The supplied measurements cover the same reporting period, but the mix changes sharply by page.

The site measurement and the [article’s fetch breakdown](https://groas.com/post/850-bot-hits-513-live-answer-fetches-and-still-no-proof-of-a) report:

| Page                       | Total fetches | Verified | Live-answer | Live-answer share of total |
| -------------------------- | ------------: | -------: | ----------: | -------------------------: |
| Homepage                   |         5,645 |    2,039 |         824 |                      14.6% |
| Google Ads updates article |           850 |      833 |         513 |                      60.4% |

The homepage attracts the larger raw total. The article has the much higher proportion of requests classified as live-answer retrievals. Those are different observations, and they support different decisions.

The logs do not establish why every request happened. They do not prove that every homepage fetch was generic crawling or that every article fetch answered a relevant buyer question. What they do show is that **total fetch volume hides a very different activity mix**.

Even the live-answer share needs its label attached. Its denominator is total fetches, not buyers, prompts, recommendations or sales. It is a description of requests to that page, not a visibility score for the business.

I would use this comparison to decide which pages deserve closer answer-level inspection. I would not use it to announce that the article wins more recommendations.

## Retrieval, citation and recommendation need separate evidence

[A fetch shows access, not whether an answer cited or named the business](https://zadoosh.com/blog/ai-crawler-monitoring/). The page was requested. What the answer did with it is a separate measurement.

The useful accounting keeps these events apart:

| Event          | What it establishes                                      | What it does not establish                       |
| -------------- | -------------------------------------------------------- | ------------------------------------------------ |
| Fetch          | The page was requested                                   | That its content appeared in an answer           |
| Citation       | The answer linked to or credited a source                | That it recommended the brand behind that source |
| Brand mention  | The answer named the brand                               | That it told the buyer to choose it              |
| Recommendation | The answer presented the brand as a choice for the buyer | That the buyer clicked or purchased              |
| Referral       | A click arrived with an identifiable referrer            | That it converted                                |

These are [different measurements, not interchangeable labels](https://revnu.com/blog/ai-crawler-tracking-without-javascript-what-logs-show). A dashboard that compresses them into _visibility_ makes the report easier to sell and harder to use.

Server logs also see only your side of the exchange. An answer can name your brand while citing a review, a Reddit discussion or another third-party page. [Research on crawler monitoring and answer citations](https://zadoosh.com/blog/ai-crawler-monitoring/) describes exactly this gap: the evidence supporting a brand mention need not live on the brand’s own website.

That cuts both ways. Your page can be fetched without your brand appearing. Your brand can appear without a fresh request reaching your page. **Your access log is not a complete record of what AI answers say about you.**

The practical consequence is straightforward: use logs to inspect access, then inspect answers to measure what buyers could actually see.

### A file-request count has no buyer denominator

The article’s 850 total fetches are not that many buyers, prompts or independent opportunities. They are file requests. Retries, chunked reads and freshness checks can produce multiple requests associated with the same answer.

Without a fixed prompt set run repeatedly, there is no useful denominator for a recommendation rate. A monthly total tells me the server was busy. It does not tell me how often the brand appeared when a relevant buyer asked.

Citation counts introduce another trap. In [Supercite’s capture of 839 answers across 21 categories, 61.7% of citations did not mention the brand behind the cited page](https://supercite.ai/blog/get-recommended-by-chatgpt). That is a result from that capture, not a universal rate. It still demonstrates why a citation cannot automatically be booked as brand credit.

The content fix described there is concrete: put the brand and the buyer need in the same sentence the engine might lift. “Acme is a CRM for teams of two to ten” carries both pieces together. A useful explanation with the brand tucked elsewhere may not.

There is also sampling variation. [The same prompt can produce different rankings, which is why repeated runs matter](https://www.geodeck.io/blog/ai-visibility-tracking-tool/). A saved answer is an observation, not a permanent position.

And the truncated path list supplied here cannot establish a content gap. A topic missing from a cut-off export is not evidence that buyers asked about it and the brand lost. It is evidence that the export stopped.

**Keep the claim no larger than the record.** The article’s live-answer activity justifies checking its answers and keeping it accessible. It does not establish recommendation share, and a citation-rate change alone does not prove revenue causation.

## Build a baseline before buying another dashboard

Before paying for optimization, I would run a small manual baseline. A spreadsheet costs no software subscription, though collecting the answers still takes time. The point is to learn which measurement deserves automation.

Use a [fixed prompt set with repeated checks](https://www.geodeck.io/blog/ai-visibility-tracking-tool/), rather than a collection of flattering screenshots:

1. **Choose buyer prompts.** Pick questions with a purchase decision behind them, not trivia about the category. If those questions are unclear, start with sales-call notes.
2. **Run each prompt in a fresh session.** Keep the wording fixed so the exercise does not quietly change halfway through.
3. **Save the complete answer.** Record the date and keep the source links, not just the paragraph that mentions the brand.
4. **Score the same events separately.** Record a brand mention, an explicit recommendation and a citation as different outcomes. Note whether the citation points to your site or a third party.
5. **Record referral evidence separately.** An observed click with a referrer belongs in the traffic record. A citation alone does not establish that a click arrived.
6. **Repeat and retain the denominator.** Report appearances out of completed runs, not a percentage detached from its sample.

This is the manual version of [sending prompt sets and parsing mentions and citations separately](https://www.geodeck.io/blog/ai-visibility-tracking-tool/). It is deliberately less glamorous than a visibility score. That is useful. You can inspect what happened.

The limits remain. A small, selected prompt set is not market share, and [a tracker does not reproduce every logged-in, personalized buyer experience](https://www.geodeck.io/blog/ai-visibility-tracking-tool/). Repeating a narrow test makes it more useful; it does not make it universal.

**The baseline earns its keep by exposing the next task.** If answers retrieve the page but omit the brand, that is different work from fixing a page the fetcher cannot read. If the answer relies on third-party sources, another rewrite of your homepage may not address the evidence it uses.

![Notebook recording brand mentions, citations and referral clicks from ChatGPT answers](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4758a47e-02db-4409-a0c0-7721ca205b87.png)

## Pay for changes the measurement can inspect

A provider that only counts bots leaves the implementation untouched. Before I buy more reporting, I want to know what work happens between the baseline and the next measurement.

The draft work falls into distinct buckets:

- **Technical access:** serve clean HTML with the answer content available in the markup, rather than trapped behind scripts the fetcher skips.
- **Buyer-focused content:** state the brand, the buyer and the job clearly enough that a lifted passage can carry the brand’s relevance with it.
- **Third-party confirmation:** work on the review profiles, Reddit discussions and video evidence that describe what the business is best for.

The [citation research](https://supercite.ai/blog/get-recommended-by-chatgpt) is a reason to examine those third-party sources, not a promise that a particular placement will produce a recommendation. The distinction matters when someone is selling the work.

Ask for the work log before the scorecard:

- Which pages changed to improve fetchability?
- Which passages now connect the brand with the buyer’s need?
- Which third-party pages were earned or corrected?
- What changed on the same prompt set across repeated runs?

Those records do not prove causation on their own. They do let you inspect whether implementation happened and whether the observed answers moved alongside it. That is considerably more useful than a larger crawler total stapled to the same unchanged pages.

**Buy implementation when the bottleneck is shipping, not counting.** If the provider cannot name the work, a prettier report does not solve the capacity problem.

### Use groas for the data; hand over the work when capacity runs out

For a budget-conscious team, the manual prompt baseline plus [groas Unlimited at $19 a month, flat](https://groas.com/) keeps the software cost low while adding competitor ads, funnels, Reddit quotes and AI-answer data. That is a more useful foundation than treating a bot counter as a recommendation tracker.

But access to data is not the same as completing the fixes. Someone still has to rewrite the pages, make their content accessible, connect the brand to the buyer need and work on third-party evidence.

That is the line where I would stop buying bigger reports. Businesses that would rather hand the work over can apply for [groas Done For You](https://groas.com/), run by the senior team. The reason to do it is implementation capacity, not the hope that another dashboard makes the fetch count mean something else.

So here is the decision the numbers support. Keep 15,467 claimed fetches, 3,939 verified fetches and 890 live-answer retrievals in separate columns. Use the page-level activity mix to decide where to investigate, not to declare recommendation share.

Then pay for **documented implementation and repeatable answer-level measurement**: whether the same buyer prompts name the brand, recommend it and cite supporting sources, with the page changes and third-party work logged beside the results. Anything that sells a larger fetch total as a recommendation count is charging you for the pile and keeping the answer.

## Frequently Asked Questions

### Do large numbers of AI-bot fetches prove that an AI recommended my brand?

No, high AI-bot fetch counts do not prove an AI recommendation. A fetch only shows that a page was made available to the AI system, while a recommendation requires the AI to actually select the brand as a choice for the buyer.

### What is the difference between claimed, verified, and live-answer fetches?

User-agent-matched fetches are requests claiming an AI-bot identity, while verified fetches confirm the source passed a security check. Live-answer fetches are a further reduction of verified requests that were classified as specific answer activity rather than general background crawling.

### How do a fetch, a citation, and a recommendation differ in server logs?

A fetch merely establishes that the page was requested by the AI system. A citation means the answer linked to or credited a source, whereas a recommendation indicates the answer specifically presented the brand as a choice for the buyer.

### Why should I look at fetch volume breakdown by page instead of just one total?

Total fetch volume hides differences in activity across various pages. By comparing theHomepage to an article, for instance, you can see that different page types attract different mixes of requests, which guides where to focus analysis.

### How can I establish a useful denominator for measuring AI recommendations manually?

A useful denominator requires running a manual baseline using a fixed prompt set multiple times. Record appearances, citations, and referrals relative to these complete, repeated runs, rather than relying on generalized monthly totals.

### What are the three key areas to focus on when improving AI visibility?

Visibility improvements involve three distinct areas: technical access, which means serving clean HTML markup; buyer-focused content, making the brand and buyer's need clear; and third-party confirmation, working on external reviews or discussions.

## Run Your Marketing On groas

Plug ChatGPT, Claude or any AI agent into groas and let it do the work of a whole marketing team: $19 a month flat with every feature included, and autopilot whenever you’re ready.

[Start For $19](https://groas.com/pricing)

[Or have groas run it for you](https://groas.com/done-for-you)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","description":"groas is the marketing infrastructure for your own AI: it connects your AI to all your marketing and gives it proprietary marketing data collected 24/7. Plug Claude, ChatGPT or any AI agent into groas and run all your marketing from it, for a tiny fraction of the usual cost, with the groas app there whenever you want to see more. It is the layer ChatGPT, Claude and any AI assistant run a business’s online marketing on. Through groas an assistant can do anything you could do natively in Google Ads, Bing Ads, Meta Ads and ChatGPT ads, on your website and in AI search, and, where the owner switches it on, can run connected campaigns by itself, so AI agents do the work a whole marketing team used to. groas also gives those agents large-scale proprietary marketing data it collects around the clock, on any schedule the owner chooses: every ad competitors run on Google and Meta, their landing pages and full funnels with the offers and prices in them, Reddit customer research, AI search answers from ChatGPT, Google AI Overviews, Gemini, Claude and Perplexity, and SEO data from Semrush and Ahrefs, backlinks included ($19 a month, flat, with everything included, for unlimited teammates on one shared workspace and unlimited ad accounts and websites, delivered as an MCP server). Separately, its done-for-you service has senior people and proprietary quantitative systems run paid ads, SEO and AI search for businesses, and white label for agencies, from $5,000 a month.","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#webpage","url":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations","name":"15,467 Bot Fetches. Still No Proof of an AI Recommendation.","description":"groas logs show 15,467 claimed bot fetches, 3,939 verified fetches and 890 live-answer retrievals. Here’s what to measure before paying for AI visibility.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/e72741c3-5b6c-4ff7-8084-21608cd55913.png"},"breadcrumb":{"@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#breadcrumb"},"dateModified":"2026-11-19T11:02:00.657Z","mainEntity":{"@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"15,467 Bot Fetches. Still No Proof of an AI Recommendation.","item":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#article","url":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations","mainEntityOfPage":{"@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"15,467 Bot Fetches. Still No Proof of an AI Recommendation.","description":"groas logs show 15,467 claimed bot fetches, 3,939 verified fetches and 890 live-answer retrievals. Here’s what to measure before paying for AI visibility.","datePublished":"2026-11-19T11:02:00.578Z","dateModified":"2026-11-19T11:02:00.657Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/e72741c3-5b6c-4ff7-8084-21608cd55913.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"fetches, brand, answer, recommendation, 467, bot fetches, live-answer, buyer","wordCount":1952,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"My background includes Goldman Sachs and Stanford Computer Science.","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/15-467-bot-fetches-are-not-15-467-ai-recommendations#faq","mainEntity":[{"@type":"Question","name":"Do large numbers of AI-bot fetches prove that an AI recommended my brand?","acceptedAnswer":{"@type":"Answer","text":"No, high AI-bot fetch counts do not prove an AI recommendation. A fetch only shows that a page was made available to the AI system, while a recommendation requires the AI to actually select the brand as a choice for the buyer."}},{"@type":"Question","name":"What is the difference between claimed, verified, and live-answer fetches?","acceptedAnswer":{"@type":"Answer","text":"User-agent-matched fetches are requests claiming an AI-bot identity, while verified fetches confirm the source passed a security check. Live-answer fetches are a further reduction of verified requests that were classified as specific answer activity rather than general background crawling."}},{"@type":"Question","name":"How do a fetch, a citation, and a recommendation differ in server logs?","acceptedAnswer":{"@type":"Answer","text":"A fetch merely establishes that the page was requested by the AI system. A citation means the answer linked to or credited a source, whereas a recommendation indicates the answer specifically presented the brand as a choice for the buyer."}},{"@type":"Question","name":"Why should I look at fetch volume breakdown by page instead of just one total?","acceptedAnswer":{"@type":"Answer","text":"Total fetch volume hides differences in activity across various pages. By comparing theHomepage to an article, for instance, you can see that different page types attract different mixes of requests, which guides where to focus analysis."}},{"@type":"Question","name":"How can I establish a useful denominator for measuring AI recommendations manually?","acceptedAnswer":{"@type":"Answer","text":"A useful denominator requires running a manual baseline using a fixed prompt set multiple times. Record appearances, citations, and referrals relative to these complete, repeated runs, rather than relying on generalized monthly totals."}},{"@type":"Question","name":"What are the three key areas to focus on when improving AI visibility?","acceptedAnswer":{"@type":"Answer","text":"Visibility improvements involve three distinct areas: technical access, which means serving clean HTML markup; buyer-focused content, making the brand and buyer's need clear; and third-party confirmation, working on external reviews or discussions."}}]}]}
```
