--- title: "Before You Pay for an AI Visibility Score, Run This 30-Day Log Test" description: "Use your own logs to separate verified AI user-fetches from bot noise, find the pages assistants request, and ask sharper questions of visibility vendors." url: "https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day" image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/3f496446-8efc-4c99-9d6f-1907772b6f01.png" published: "2026-11-02T06:38:10.292Z" modified: "2026-11-02T06:38:10.353Z" --- November 2, 2026 · 11 min read # Before You Pay for an AI Visibility Score, Run This 30-Day Log Test [Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/) ![Clay doorman with a magnifying glass inspects a queue all wearing 'ChatGPT-User' stickers as one visitor's fake moustache falls off, exposing an impostor.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/3f496446-8efc-4c99-9d6f-1907772b6f01.png) In this article 1. [The question: which pages get a live AI fetch?](#the-question-which-pages-get-a-live-ai-fetch) 2. [Setup: collect four things before day one](#setup-collect-four-things-before-day-one) 3. [Protocol: run the same checks for 30 days](#protocol-run-the-same-checks-for-30-days) 4. [Measurements: put four views on one sheet](#measurements-put-four-views-on-one-sheet) 5. [Interpretation: what would change your next move?](#interpretation-what-would-change-your-next-move) 6. [Answer panel: check what surfaced](#answer-panel-check-what-surfaced) 7. [Vendor check: take your sheet into the pitch](#vendor-check-take-your-sheet-into-the-pitch) 8. [Rerun: make the next 30 days comparable](#rerun-make-the-next-30-days-comparable) A page can rack up AI-bot hits without an assistant fetching it for a live question even once. Before I pay for an AI visibility score, I want to know which of those hits came from a real vendor, which came from a user-fetch agent, and which pages it requested. That is why we are taking a second look at [_One Changelog Drew 377 Live AI Fetches. Three Busy Hubs Drew Zero._](https://groas.com/post/30-days-of-ai-bot-logs-on-our-own-site-1). Our first look covered one site in one month. Your site will behave differently. This time, I am giving you the test so you can inspect your own logs before paying anyone to tell you what they mean. Semrush’s AI Visibility Toolkit, for example, lists a $99-per-month-per-domain price, billed annually, for tracking 25 custom prompts across ChatGPT, Google AI, Gemini and Perplexity ([pricing](https://www.semrush.com/pricing/ai/)). A prompt panel can show what appears in answers. It cannot, by itself, show whether an assistant fetched _your_ page. Your logs can answer that narrower question. Run both measurements, and you have a better basis for deciding what work is worth buying. ## The question: which pages get a live AI fetch? **Which pages do user-fetch agents request, and which requests can I verify as coming from the claimed vendor?** Keep both halves of that question. A training crawler visiting a page is not the same event as an agent requesting it during a person’s answer session. A request merely claiming to be ChatGPT-User is not proof that OpenAI sent it. Set up three per-page counts before day one: | Measure | What it counts | What it does **not** establish | | ------------------------ | ---------------------------------------------------------------------- | ------------------------------------------------- | | Raw hits | Requests claiming an AI-bot user agent | That the requester was that bot | | Verified hits | Claimed requests whose IP matches a published vendor range | That the page appeared in an answer | | Verified user-fetch hits | Matched requests claiming ChatGPT-User, Claude-User or Perplexity-User | That the answer named, quoted or linked your page | **The third count is the working measure, not the outcome.** A user-fetch request puts a page closer to a live answer than a training crawl does. It still does not tell you whether the answer used that page. An AirOps study cited in [AuthorityTech’s Perplexity breakdown](https://authoritytech.io/blog/how-perplexity-selects-sources-algorithm-2026) ran 15,000 prompts that led ChatGPT to retrieve 548,534 pages; 15% of retrieved pages made the final response. A fetch gets a page into the room. It does not give it the part. ### Keep the blind spots on the sheet Fetch is not citation. Citation is not a recommendation, and a recommendation is not a click. Your logs cannot reveal the question that caused a fetch, whether the assistant named you, or what the person did next. They also cannot see a competitor’s logs or an answer generated without fetching your site. In a 20-question test reported by [Foglift Research](https://geoaura.world/blog/chatgpt-search-citation-mechanism), ChatGPT answered 8 of 20 from memory with no citations, while Perplexity cited on 19 of 19. **Record an unseen event as unknown, not zero.** A prompt panel observes answers from the other side; it has blind spots too. You will use the two instruments together, not pretend either one sees the whole process. ## Setup: collect four things before day one 1. **Request-level logs with a full 30-day window.** CDN, edge or server logs will work if they retain each request’s timestamp, path, user agent and client IP. Aggregated pageviews will not. Check retention first: a platform keeping only seven to fourteen days will leave you with a shorter test. If you use origin logs, note whether CDN caching can keep requests from reaching them. 2. **An agent list separated by job.** Track training crawlers such as GPTBot, ClaudeBot and PerplexityBot; indexing crawlers such as OAI-SearchBot and Googlebot; and user-fetch agents such as ChatGPT-User, Claude-User and Perplexity-User. [Anagram’s crawler guide](https://www.anagram.ai/blog/ai-crawlers-explained-gptbot-claudebot-perplexitybot-and-how-to-let-them-in-2026) lays out the distinction. Google-Extended does not have its own user agent, so do not count it as a distinct visitor. The practitioners in [r/aeo](https://www.reddit.com/r/aeo/comments/1w3dyou/how_to_check_which_ai_assistants_are_reading_your/) also flag what logs cannot show. Leave an explicit gap for agents you cannot identify or verify. 3. **The vendors’ published IP ranges.** A user-agent string is a header anyone can claim. [eSEOspace’s verification guide](https://eseospace.com/blog/verify-ai-bots-in-server-logs/) describes OpenAI’s separate files by agent, Anthropic’s combined list and Perplexity’s two lists. Match the client IP against the relevant file, retaining the agent claimed in the request. If your export omits client IPs, stop: you cannot perform this check. [Machine Relations’ audit](https://machinerelations.ai/research/ai-crawler-user-agent-verification-gap-2026) makes that limitation hard to ignore. 4. **A comparison sheet.** Give each path columns for raw hits, verified hits, verified user-fetch hits and agent type. Add space for your top human-traffic pages and, later, a small panel of buyer questions. This prevents a busy crawler from winning the test by volume alone. ## Protocol: run the same checks for 30 days ### Week 1: capture requests; keep the failures Export requests carrying the AI-bot user agents on your list. For each row, retain the timestamp, path, claimed agent and client IP. Match the IP to the relevant vendor range, then label the row **matched**, **failed verification** or **no usable vendor range**. Do not delete failed rows. The gap between claimed and matched traffic is part of your result. Also separate content paths from `/robots.txt` and `sitemap.xml`. A five-day sample published on [r/webscraping](https://www.reddit.com/r/webscraping/comments/1x4sxlp/monitoring_bot_access_of_my_sites/) illustrates why: its 59 ClaudeBot requests went only to those two files, not to a content page. Counting all 59 as article reads would make a permissions check look like an audience. ### Weeks 2 and 3: classify by agent and page Tag matched requests as training, indexing or user-fetch, then group them by path and agent. **Keep separate columns for each user-fetch agent.** A page with many raw hits but no matched user-fetch hits has a different story from one with modest raw traffic and repeated matched user-fetch requests. Record each page’s matched share alongside its totals. Do not call that share an answer rate: it measures how much claimed traffic you could confirm against the published ranges. For the question this test asks, your main page-level result is the count of matched user-fetch requests. You can add a prospective observation here: publish one dated, answer-first _what changed_ page on a question your buyers ask, and tag its URL in the sheet. Put the change and whom it affects in the opening paragraph. Our changelog’s 377 verified user-fetch hits make that structure worth testing on your site. **Do not call one new page a controlled experiment.** Its topic, timing and format all change together, and it has no control twin. It can suggest a page pattern; it cannot isolate what caused the fetches. ### Week 4: freeze the window and write down the gaps Export the complete 30-day window and reconcile it with your earlier pulls. Check for missing IPs, shortened retention, unmatched agents, path variants created by query strings, and CDN requests your origin logs may not contain. Keep those gaps beside the results. A dashboard that hides them looks cleaner. Your test will be more useful. ## Measurements: put four views on one sheet **1. Matched share of AI-claimed requests.** Divide matched hits by raw hits, by agent and path. In groas’s logs, roughly 70% of bot-labelled traffic on many paths failed verification; a [separate groas audit](https://groas.com/post/76-of-our-ai-bot-traffic-couldn-t-be-ver) found that 76.5% of claimed AI-bot hits could not be confirmed. Your share may differ. Treat it as a warning against counting strings as verified visits, not as a discount you can apply to every kind of visibility score. **2. User-fetch requests per page and assistant.** Break out matched ChatGPT-User, Claude-User and Perplexity-User requests rather than blending them into one AI number. Answer behaviour differs too: [Foglift’s analysis](https://geoaura.world/blog/chatgpt-search-citation-mechanism) reported averages of 3.29 citations per ChatGPT answer and 9.68 per Perplexity answer across 61,680 answers. The log count still cannot tell you which of your pages was cited. The split tells you where to look next. **3. Overlap with your human-traffic leaders.** Export your top 20 pages by organic sessions from GA4 or Search Console. Mark which appear in the matched user-fetch table. A product page high on the human list and absent from the fetch list deserves inspection, not an automatic rewrite. [Asked Thrice’s citation guide](https://askedthrice.com/blog/how-to-get-cited-by-chatgpt-and-perplexity.md) describes its own 60-day sample of 505 live buyer answers and 438 cited URLs, most of them home pages. Your overlap check tells you whether your site shows a similarly lopsided pattern. **4. Robots checks by agent.** Count requests to `/robots.txt` per agent per day, but keep them out of content-page totals. They show permission-check activity, not article reads. If the pattern changes, investigate it; do not promote your robots file to your best-performing post. ## Interpretation: what would change your next move? ### If claimed AI hits mostly fail verification **Stop treating a user-agent count as a count of verified vendor visits.** Ask any tool reporting bot traffic whether it retains client IPs and matches them to published ranges. Machine Relations sampled 3,540 requests from 59,166 claiming AI-crawler user agents and found 27% did not match a vendor-published range ([its audit](https://machinerelations.ai/research/ai-crawler-user-agent-verification-gap-2026)). That does not tell you what share will fail on your site. Your test does. A prompt-based mention score is a different measurement; do not subtract your failed-verification share from it. Ask how the vendor collected its answers instead. Same label, different instrument. ### If live fetches cluster on a few pages Read the pages before making a content plan. Do the winners answer a specific question early, carry a useful date, or provide facts that stand on their own? In our logs, the changelog had that shape; the busy hubs were link menus with general summaries. **A recurring page pattern is worth another test; a single winner is not a template yet.** If the pages share nothing you can identify, keep collecting data rather than naming a formula. ### If product pages barely appear Rule out a delivery problem before rewriting copy. A bot requesting server HTML may not receive content injected after JavaScript runs. If the price or availability detail appears only after hydration, inspect the raw response with `curl` and see what is there ([Stackademic’s log study](https://blog.stackademic.com/what-ai-crawlers-actually-do-to-a-small-blog-9-days-of-logs-37ea59c7a7ae?gi=7540288576ac) discusses the gap). Then inspect specificity. In our data, broad ‘flexible solution for modern teams’ copy got crawled and skipped, while pages putting priced tiers, named integrations and dated availability up front drew some user-fetches. **Put a checkable fact where the answer begins, then measure the next window.** That is a hypothesis you can test, not proof that wording alone caused the difference. ### If your new dated page gets fetched The useful finding is that the page attracted matched user-fetch requests. It does not prove the date caused them or that an answer cited the page. Your next check is independent: run the question that page answers through an answer panel and record whether the brand or page appears. **Fetches rising while mentions stay flat is a different result from both rising together.** In the first case, inspect what the assistant says; in the second, keep testing that page shape. Neither case calls for a victory slide after one page. ## Answer panel: check what surfaced Take pages with matched user-fetch requests and write the buyer question each answers. Not the keyword: the question. Our changelog addresses what changed in Google Ads this year and what to do about it. A pricing page addresses what something costs. Run the same questions through ChatGPT, Gemini, Perplexity and Google’s AI Overviews on the same day, then repeat every two weeks. For each answer, record **brand named, page cited, competitor cited, or no source shown**. Keep the prompt wording and schedule stable. Fifteen consistently repeated prompts tell me more about movement than two hundred prompts run once. The panel does not identify which logged request belongs to which answer. It does show whether the pages drawing fetches also surface for the questions you care about. Logs tell you what reached your site; the panel tells you what a reader saw. That is the comparison a single score skips. ## Vendor check: take your sheet into the pitch Ask a visibility tool or agency three questions: 1. **For bot-traffic claims, do you verify IPs or count user-agent strings?** Ask to see how failures and unverifiable requests appear in the report. 2. **Do you separate training, indexing and user-fetch activity?** One AI-traffic line cannot answer this test’s question. 3. **For answer claims, which prompts, assistants, pages and collection dates can I inspect?** A mention without those details is difficult to act on. Tools can disagree even before you get to strategy. In [Roberto Serra’s three-month comparison](https://www.roberto-serra.com/en/geo/measuring-ai-visibility/tools/ai-visibility-tracking-tool-i-tested-peec-otterly-profound-3-months/), Otterly confirmed four mentions Peec found and identified two more; Profound was oversized for that client’s four to six mentions in three months. His advice was to measure by hand before paying. This protocol gives that work a 30-day window and adds a verification check. Watch for claims that a mention score proves a page was fetched or read. It does not. An [AILabsAudit review](https://ailabsaudit.com/blog/en/ai-visibility-barometer-brands-2026) found non-existent URLs among AI citations; a cited link is not automatically a working page. Ask for the underlying answer and check the URL yourself. **My bar is inspectable evidence before a polished score.** That is where groas’s SEO and AI Search service is a better fit than paying for a score alone: [groas](https://groas.com/pricing) pairs search work with a named strategist and a flat monthly fee, rather than leaving you to turn a dashboard number into an action plan. Bring the sheet either way. A vendor should be able to tell you which instrument supports each claim and what they will do with the result. ## Rerun: make the next 30 days comparable Run the test quarterly using the same log source, agent categories and prompt panel. Note seasonal changes before comparing windows. On a small site, treat single-digit differences as a reason to keep watching, not to declare a winner. **If the pages you expect to win get verified user-fetches and appear in answers, keep testing their pattern. If they get fetched but never surface, inspect the answers and the pages. If they get neither, check access and rethink the questions those pages answer.** Either result gives you a next move. A score without that distinction gives you another number to put in a deck. ## Frequently Asked Questions ### How do I differentiate between a training crawl and a live AI user fetch in my logs? Your logs count three distinct measures: raw hits, verified hits, and verified user-fetch hits. Raw hits are any request claiming an AI-bot user agent. Verified hits are those matched to a published vendor IP range. Verified user-fetch hits are matched requests specifically claiming user agents like ChatGPT-User. ### How can I verify if an AI-bot request actually belongs to the vendor? A user agent claim is not sufficient proof of origin. To verify a request, the client IP must be matched to the published range of the claimed vendor. Without client IP data, you cannot perform this verification check. ### What specific data must I collect before starting a 30-day AI log test? You need 30 days of request-level logs that include timestamps, paths, user agents, and client IPs. You must also compile a list of AI agents separated by job (training, indexing, user-fetch) and acquire the vendors' published IP ranges. ### What limitations exist when analyzing AI visibility using only my website logs? Website logs cannot reveal the question that caused a fetch, whether an assistant named your brand, or what a person did after seeing an answer. Logs only show what reached your server, not the final outcome or citation details. ### If my page receives a verified user-fetch hit, what should I do next? A fetch indicates a page is relevant to a live question but does not prove it was cited. The next step is to run the question that page answers through an answer panel and check if your brand or page appears in the resulting AI answer. ### What three questions should I ask a visibility tool provider? Ask if the tool verifies IPs rather than just counting user-agent strings. Confirm that the tool separates training, indexing, and user-fetch activity. Finally, request documentation detailing which prompts, assistants, and collection dates you can inspect. ## Paid Ads & SEO, Run For You Businesses pay one flat monthly fee per service, agencies resell it under their own brand, and groas does the work either way. [See Pricing](https://groas.com/pricing) ## Structured data ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","description":"groas is a done-for-you marketing service, not software: elite people working with proprietary technology systems run search ads, Meta ads, SEO and AI search for businesses, and white label for agencies, with a dedicated senior strategist owning every account.","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#webpage","url":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day","name":"Before You Pay for an AI Visibility Score, Run This 30-Day Log Test","description":"Use your own logs to separate verified AI user-fetches from bot noise, find the pages assistants request, and ask sharper questions of visibility vendors.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/3f496446-8efc-4c99-9d6f-1907772b6f01.png"},"breadcrumb":{"@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#breadcrumb"},"dateModified":"2026-11-02T06:38:10.353Z","mainEntity":{"@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Before You Pay for an AI Visibility Score, Run This 30-Day Log Test","item":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#article","url":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day","mainEntityOfPage":{"@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Before You Pay for an AI Visibility Score, Run This 30-Day Log Test","description":"Use your own logs to separate verified AI user-fetches from bot noise, find the pages assistants request, and ask sharper questions of visibility vendors.","datePublished":"2026-11-02T06:38:10.292Z","dateModified":"2026-11-02T06:38:10.353Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/3f496446-8efc-4c99-9d6f-1907772b6f01.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"page, requests, pages, test, agent, hits, user-fetch, logs","wordCount":2358,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"My background includes Goldman Sachs and Stanford Computer Science.","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/before-you-pay-for-an-ai-visibility-score-run-this-30-day#faq","mainEntity":[{"@type":"Question","name":"How do I differentiate between a training crawl and a live AI user fetch in my logs?","acceptedAnswer":{"@type":"Answer","text":"Your logs count three distinct measures: raw hits, verified hits, and verified user-fetch hits. Raw hits are any request claiming an AI-bot user agent. Verified hits are those matched to a published vendor IP range. Verified user-fetch hits are matched requests specifically claiming user agents like ChatGPT-User."}},{"@type":"Question","name":"How can I verify if an AI-bot request actually belongs to the vendor?","acceptedAnswer":{"@type":"Answer","text":"A user agent claim is not sufficient proof of origin. To verify a request, the client IP must be matched to the published range of the claimed vendor. Without client IP data, you cannot perform this verification check."}},{"@type":"Question","name":"What specific data must I collect before starting a 30-day AI log test?","acceptedAnswer":{"@type":"Answer","text":"You need 30 days of request-level logs that include timestamps, paths, user agents, and client IPs. You must also compile a list of AI agents separated by job (training, indexing, user-fetch) and acquire the vendors' published IP ranges."}},{"@type":"Question","name":"What limitations exist when analyzing AI visibility using only my website logs?","acceptedAnswer":{"@type":"Answer","text":"Website logs cannot reveal the question that caused a fetch, whether an assistant named your brand, or what a person did after seeing an answer. Logs only show what reached your server, not the final outcome or citation details."}},{"@type":"Question","name":"If my page receives a verified user-fetch hit, what should I do next?","acceptedAnswer":{"@type":"Answer","text":"A fetch indicates a page is relevant to a live question but does not prove it was cited. The next step is to run the question that page answers through an answer panel and check if your brand or page appears in the resulting AI answer."}},{"@type":"Question","name":"What three questions should I ask a visibility tool provider?","acceptedAnswer":{"@type":"Answer","text":"Ask if the tool verifies IPs rather than just counting user-agent strings. Confirm that the tool separates training, indexing, and user-fetch activity. Finally, request documentation detailing which prompts, assistants, and collection dates you can inspect."}}]}]} ```