---
title: "Stop Guessing Which AI Engine to Fix: A 30-Day Visibility Test"
description: "Run the same buyer prompts in ChatGPT, Perplexity and Google AI Overviews, pair the answers with server logs, and test whether one focused fix changes what buyers see."
url: "https://groas.com/post/measure-before-you-optimize-a-30-day-tes"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/641cdfbc-8202-4287-b3ab-16af4a92cd69.png"
published: "2026-10-25T05:25:06.974Z"
modified: "2026-10-25T05:25:07.050Z"
---

October 25, 2026 · 11 min read

# Stop Guessing Which AI Engine to Fix: A 30-Day Visibility Test

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A mechanic logs results on a clipboard in front of three different engines with identical test tags, her wrench untouched: testing before fixing.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/641cdfbc-8202-4287-b3ab-16af4a92cd69.png)

In this article

1. [Question: which engine shows you, cites you or fetches you?](#question-which-engine-shows-you-cites-you-or-fetches-you)
2. [Setup: write 20 prompts a buyer might actually use](#setup-write-20-prompts-a-buyer-might-actually-use)
3. [Measure 1: record the answer, not the screenshot you liked](#measure-1-record-the-answer-not-the-screenshot-you-liked)
4. [Measure 2: check which pages the bots request](#measure-2-check-which-pages-the-bots-request)
5. [Measure 3: treat AI Overviews as their own condition](#measure-3-treat-ai-overviews-as-their-own-condition)
6. [The 30-day schedule: baseline, one change, retest](#the-30-day-schedule-baseline-one-change-retest)
7. [Read the result without turning variance into a victory](#read-the-result-without-turning-variance-into-a-victory)
8. [Choose the next fix from the evidence](#choose-the-next-fix-from-the-evidence)
9. [Repeat monthly, or admit the result did not hold](#repeat-monthly-or-admit-the-result-did-not-hold)

Most advice about which AI engine to fix first is a guess dressed up as a priority list. I wrote an earlier starting point in [ChatGPT vs Perplexity vs Google AI Overviews: Which Citation Path Should You Fix First?](https://groas.com/post/chatgpt-vs-perplexity-vs-google-ai-overv-2). We are taking a second look because the right starting point depends on how each engine already treats your site, and that varies from one business to the next.

This time, run a test. Ask the same buyer questions in three engines on a schedule, record the answers and check them against your own server logs. Make one fix, then ask again. After 30 days, the priority should come from your numbers rather than my ranking.

## Question: which engine shows you, cites you or fetches you?

For a fixed set of buyer prompts, how often does your brand appear in ChatGPT, Perplexity and Google AI Overviews? When does your page appear as a source, and which engines fetch pages from your site?

Keep the events separate. **A mention is not a citation, and a citation is not necessarily a clickable link.** A mention puts your name in the answer. A citation identifies your page as a source. A link gives the reader a route to it. Your log files answer a different question: whether a bot requested a page. A fetch alone does not tell you that the page appeared in an answer.

My expectation is that your results will not line up neatly across the three engines. One may fetch pages it rarely cites; another may show your brand on prompts where its bot barely appears in your logs. That mismatch is useful. It tells you whether to investigate access, the pages being selected or the answer the buyer actually sees. It also makes a general ranking a poor substitute for a baseline.

The limit matters, too. This is **a controlled sample, not a count of your total AI exposure**. Engines do not give you full impression data, and [Google Search Console and GA4 cannot isolate AI Overview traffic](https://searchatlas.com/blog/ai-overviews/) in the way this test needs. Your sheet will measure the prompts you chose under the conditions you kept steady. Do not sell it to yourself as every answer every buyer saw.

## Setup: write 20 prompts a buyer might actually use

Start with 20 questions from sales call notes, support tickets and Search Console queries. Do not turn category-page headlines into questions and call that buyer research. I used to write these from what I thought buyers _should_ ask. I was wrong. Use the language someone types when they need a price, a comparison, a local option or a fix for a specific problem.

Split the set into three rough groups so you can interpret the answers later:

- **Category questions:** Which type of product or service fits a stated need?
- **Comparison questions:** How does one option stack up against another for that need?
- **Problem questions:** What is going wrong, what might fix it and what might it cost?

Practitioners using a [manual citation-tracking method](https://discoveredlabs.com/blog/google-ai-overviews-citation-tracking-how-to-measure-your-visibility) work with 20 to 50 buyer questions, run them weekly and log whether an Overview triggered and who got cited. For this test, keep the core 20 unchanged through the retest. If you quietly improve the prompts in week four, you have changed the instrument along with the site.

Set the controls before your first run. Use fresh temporary chats, stay logged out where you can, keep location and language settings fixed, and use the same prompt wording on the same day across ChatGPT with browsing on, Perplexity and Google. For Google, record an absent AI Overview as an absent Overview. Do not swap in a different question until one appears.

Why repeat the prompts? Identical questions do not reliably produce identical answers. In one check, repeated ChatGPT prompts [shared only about 23% of businesses named](https://medium.com/@blogchoice/does-chatgpt-recommend-your-business-396fa7835b40). Another teardown found that the same short prompt, run minutes apart, [shared zero exact retrieval queries](https://www.seerinteractive.com/insights/chatgpt-data-structure); it retrieved 14 URLs on one run and 40 on another. One screenshot is an observation, not your visibility score.

Make the sheet **one row per prompt, per engine, per run**. Include:

- Prompt, prompt group, date, engine and repeat number.
- Whether the answer mentions your brand, cites your page or links to it.
- The exact URL shown, every competitor cited and the total citations shown.
- A screenshot or saved answer so you can check the row later.
- For Google, whether an AI Overview triggered at all.

Keep mentions, citations and links in separate columns. If a result looks odd next month, you should be able to reopen the evidence rather than debate what someone remembers seeing. Above all, log the URL. A tick beside ‘cited’ will not tell you which page needs work.

## Measure 1: record the answer, not the screenshot you liked

Run each of the 20 prompts five times per engine in fresh sessions. Log every answer, including the ones where you are absent. That gives you 100 attempts per engine in a testing round, although Google will not necessarily show an Overview on all 100. Do not compare its Overview-only citations with 100 ChatGPT answers without naming the different denominators.

For each engine, calculate **mention rate as answers naming you divided by answers tested**. Track citation rate separately using answers that cite your page divided by answers tested. For Google, also calculate citations among the answers where an Overview actually triggered. Put both numbers beside the trigger count; otherwise a respectable citation rate inside four Overviews can hide the fact that most of your tested queries never produced one.

Then read the URLs, not just the percentages. A homepage citation on a brand question does not tell you whether a relevant page appears on a comparison question. Your sheet might show the same guides repeatedly while another page never appears. That pattern gives you a place to investigate during the change window. Rewriting the whole site would give you less information.

Expect movement between repeats. In the retrieval teardown, 34 of 40 pages fetched in one run were not cited. Retrieval and citation are different decisions. If your brand appears once in five runs for a prompt, note it, but do not build a month of work around it. If it appears four or five times under the same controls, you have a pattern worth testing again.

## Measure 2: check which pages the bots request

Answers show what you observed as a reader. Server or CDN logs show requests to your site. Pull the 30 days before you start and filter for `ChatGPT-User`, `OAI-SearchBot`, `GPTBot`, `PerplexityBot` and `Perplexity-User`. Save the period and the filter you used so the later pull is comparable. **Sort by bot and URL before you decide which page to edit.**

Do not lump those user agents together. [OpenAI documents separate controls for its bots](https://developers.openai.com/api/docs/bots/): `OAI-SearchBot` is for surfacing sites in ChatGPT search, `GPTBot` is for training, and `ChatGPT-User` handles user-triggered fetches. Search access and training access are different choices. [Perplexity also distinguishes](https://docs.perplexity.ai/guides/bots) `PerplexityBot`, which surfaces and links sites in results, from `Perplexity-User`, which fetches for a user question and generally ignores robots.txt.

Check your robots.txt rules against the bot relevant to the path you are testing. Blocking `GPTBot` does not, by itself, tell you what you allowed `OAI-SearchBot` to do. Equally, a bot request does not prove that your page was used in a visible answer. Treat the logs as evidence of requests, then compare them with your answer rows. A page requested often but never cited calls for a different investigation from a page the relevant bot does not request at all.

For the post-change comparison, use matching windows and report the window beside the count. If your baseline is a full 30 days and your retest is one week, raw fetch totals will mainly tell you that one period lasted longer. Keep the earlier 30-day pull for context; use comparable slices to judge the change.

## Measure 3: treat AI Overviews as their own condition

You cannot isolate an AI Overview with a separate fetcher count in these logs, so measure that part of the test from the answer side. For every Google run, record whether an Overview appears. When it does, expand it, capture every cited source and save the expanded screenshot. Keep region and language fixed. Use an incognito window and clear cookies between queries.

The [manual logging approach](https://discoveredlabs.com/blog/google-ai-overviews-citation-tracking-how-to-measure-your-visibility) separates ‘AI Triggered’, ‘Our Brand Cited’, ‘Competitor Cited’ and ‘Total Citations’. Do the same. Count link cards in the citation column, not plain-text brand mentions. A text mention may be worth recording, but it is not the same event as a source the reader can inspect.

Read trigger rate alongside citation rate. Suppose four of your 20 prompts trigger an Overview and you appear in one. That is one citation among four triggered prompts, but only one across the full prompt set. Neither number is wrong; each answers a different question. Preserve both before you choose a fix.

## The 30-day schedule: baseline, one change, retest

A protocol is only useful if you can tell what changed. Put these dates in the sheet before anyone edits a page.

1. **Week 1 — baseline.** Run all 20 prompts in all three engines, five repeats each. Pull and save the preceding 30 days of bot logs. Record your settings and take your screenshots. Make no site changes during the answer-side baseline.
2. **Weeks 2 and 3 — one fix.** Pick a single change that follows from the baseline. If the relevant search bot is blocked, review that rule. If a fetched page is repeatedly passed over, consider a concise answer passage with the date and number the question needs, or update stale specifications. Record the page, the exact change and the date it went live. Do not also publish a batch of new pages and rewrite every comparison page.
3. **Week 4 — retest.** Run the same prompts, in the same engines, with the same repeat count and controls. Pull the new logs. Compare answer rates from the two testing rounds, then compare bot requests over matched log windows. Note if the change was not live for all of a log window rather than treating every request in it as post-fix.

You will still have uncertainty. Engines vary, buyers vary and a month is short. But **one documented change beats five simultaneous ‘optimizations’** when the point is to learn which action moved an observable number.

## Read the result without turning variance into a victory

Small samples lie with confidence. One appearance in five runs is a 20% rate for that prompt, but it is also one appearance. I treat one of five as a reason to keep watching, two or three as an emerging pattern, and four or five as something to retest. Across the full set, show the counts behind every percentage. That is more honest than the single-run scores in the sort of [visibility reports I have pushed back on before](https://groas.com/post/7-myths-about-measuring-your-brand-s-vis).

The same restraint applies after your fix. If citation rate moves from 10% to 15% on the same prompts, do not declare a win; look at the individual rows and repeat next month. A move from 10% to 45% deserves attention, but first check that the settings, trigger counts and scoring rules stayed the same. Your control sheet should make that check boring. Boring is good here.

![Lab notebook sheet tracking AI citations and bot fetches across three engines](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/f652bbd6-cab2-4cac-9e7f-74aaa2deb622.png)

Read answer and log results as a pair. If requests rise while citations stay flat, the bot reached the site but your page still did not appear as a source in this sample. Inspect the questions and pages being selected before changing access rules again. If citations appear while requests in your chosen window stay flat, do not claim a fresh fetch caused them. Record the mismatch and keep measuring.

For a simple engine-level score, count the prompts that cite you in at least three of five repeats and divide by 20. Keep the underlying answer-level rate beside it: the prompt score shows breadth; the answer-level rate shows how often citations appeared. For Overviews, you can also report [AI Share of Voice as brand citations divided by Overviews triggered](https://discoveredlabs.com/blog/google-ai-overviews-citation-tracking-how-to-measure-your-visibility), using the same counting rule each month. Label every denominator. ‘25%’ without one belongs on a sales slide, not in this notebook.

## Choose the next fix from the evidence

Now choose a priority rather than inheriting one. Start where your baseline shows a plausible opening and your sheet names a specific obstacle: a relevant page gets requested but not cited, a comparison prompt repeatedly names competitors instead of you, or a search bot cannot reach the page you intend it to use. An engine with no mentions and almost no relevant requests may need an access check, but do not spend the next month polishing its copy on instinct alone.

That is a more useful decision than ‘work on ChatGPT first’ or ‘AI Overviews first’. Once you have picked the engine, the [engine-by-engine comparison](https://groas.com/post/chatgpt-vs-perplexity-vs-google-ai-overv-3) can help you decide where to work. Use the part that fits your observed problem. **One engine, one fix, one retest.**

![Ink cartoon of a marketer choosing one of three engine doors based on test scores](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/3ff59081-391c-4c9e-8c46-f4c8d0a2858d.png)

## Repeat monthly, or admit the result did not hold

Keep the same 20 core prompts and controls next month. Add dated rows rather than replacing your old results, so you can see whether a change lasts. If buyers start asking something new, add those prompts to a separate set; do not quietly fold them into the original score. Retire a core prompt only with a note explaining why. Otherwise your trend will measure your editing habits as much as your visibility.

If citations rise and hold for two cycles, you have a stronger reason to keep working on that engine. If requests rise but citations do not, access may have improved while selection has not. If neither moves, reconsider the fix or the priority. A flat result is still a result: it tells you not to pay for more of the same work just because a report calls it progress.

That is what this 30-day test changes. It will not reveal every AI answer about your business. It will tell you what happened on questions your buyers ask, whether the relevant bots requested your pages, and whether one focused fix changed the answers in front of you. groas runs answer-side and log-side checks continuously for clients; the sheet version gives you the same discipline by hand. Measure first. Fix second. See if it held.

## Frequently Asked Questions

### What is the difference between a mention, a citation and a link in AI answers?

A mention puts your brand name in the answer, a citation identifies your page as a source, and a link gives the reader a route to that page. These are separate events: a citation is not necessarily a clickable link, and a bot fetching a page does not prove the page appeared in an answer.

### How do I choose the buyer prompts for an AI visibility test?

Write 20 questions from sales call notes, support tickets and Search Console queries, using the language buyers actually type when they need a price, comparison, local option or fix. Split them into category, comparison and problem questions, then keep the core 20 unchanged throughout the retest.

### Why run the same AI prompts multiple times instead of just once?

Identical questions do not reliably produce identical answers. One check found repeated ChatGPT prompts shared only about 23% of the businesses named, and another found the same short prompt retrieved 14 URLs in one run and 40 in another. One screenshot is an observation, not your visibility score.

### How do I calculate mention rate and citation rate in AI answers?

Run each of the 20 prompts five times per engine, then calculate mention rate as answers naming you divided by answers tested, and citation rate as answers citing your page divided by answers tested. For Google, also calculate citations among the answers where an AI Overview actually triggered, and keep the trigger count beside it.

### Which AI bots should I look for in my server logs?

Filter your logs for ChatGPT-User, OAI-SearchBot, GPTBot, PerplexityBot and Perplexity-User. Do not lump them together: OAI-SearchBot surfaces sites in ChatGPT search, GPTBot is for training, ChatGPT-User handles user-triggered fetches, and PerplexityBot surfaces sites in results while Perplexity-User fetches for a user question and generally ignores robots.txt.

### What is the 30-day schedule for testing one AI fix?

Week 1 is the baseline: run all 20 prompts five times in each of the three engines and pull 30 days of bot logs. Weeks 2 and 3, make one documented change. Week 4, retest with the same prompts and controls, then compare answer rates and bot requests over matched log windows.

### How many repeats do I need before treating a result as a real pattern?

Treat one appearance in five runs as a reason to keep watching, two or three as an emerging pattern, and four or five as something to retest. After a fix, a citation rate move from 10% to 15% is not a win on its own, so check the individual rows and repeat next month.

### What does it mean if bot requests rise but citations stay flat?

The bot reached your site but the page still did not appear as a source in the answers you sampled. Inspect the questions and pages being selected before changing access rules again. The reverse, citations with flat requests, means you should not claim a fresh fetch caused them.

## Paid Ads & SEO, Run For You

Businesses pay one flat monthly fee per service, agencies resell it under their own brand, and groas does the work either way.

[See Pricing](https://groas.com/pricing)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","description":"groas is a done-for-you marketing service, not software: elite people working with proprietary technology systems run search ads, Meta ads, SEO and AI search for businesses, and white label for agencies, with a dedicated senior strategist owning every account.","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#webpage","url":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes","name":"Stop Guessing Which AI Engine to Fix: A 30-Day Visibility Test","description":"Run the same buyer prompts in ChatGPT, Perplexity and Google AI Overviews, pair the answers with server logs, and test whether one focused fix changes what buyers see.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/641cdfbc-8202-4287-b3ab-16af4a92cd69.png"},"breadcrumb":{"@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#breadcrumb"},"dateModified":"2026-10-25T05:25:07.050Z","mainEntity":{"@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Stop Guessing Which AI Engine to Fix: A 30-Day Visibility Test","item":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#article","url":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes","mainEntityOfPage":{"@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Stop Guessing Which AI Engine to Fix: A 30-Day Visibility Test","description":"Run the same buyer prompts in ChatGPT, Perplexity and Google AI Overviews, pair the answers with server logs, and test whether one focused fix changes what buyers see.","datePublished":"2026-10-25T05:25:06.974Z","dateModified":"2026-10-25T05:25:07.050Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/641cdfbc-8202-4287-b3ab-16af4a92cd69.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"fix, page, prompts, engine, answer, test, run, answers","wordCount":2411,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"My background includes Goldman Sachs and Stanford Computer Science.","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/measure-before-you-optimize-a-30-day-tes#faq","mainEntity":[{"@type":"Question","name":"What is the difference between a mention, a citation and a link in AI answers?","acceptedAnswer":{"@type":"Answer","text":"A mention puts your brand name in the answer, a citation identifies your page as a source, and a link gives the reader a route to that page. These are separate events: a citation is not necessarily a clickable link, and a bot fetching a page does not prove the page appeared in an answer."}},{"@type":"Question","name":"How do I choose the buyer prompts for an AI visibility test?","acceptedAnswer":{"@type":"Answer","text":"Write 20 questions from sales call notes, support tickets and Search Console queries, using the language buyers actually type when they need a price, comparison, local option or fix. Split them into category, comparison and problem questions, then keep the core 20 unchanged throughout the retest."}},{"@type":"Question","name":"Why run the same AI prompts multiple times instead of just once?","acceptedAnswer":{"@type":"Answer","text":"Identical questions do not reliably produce identical answers. One check found repeated ChatGPT prompts shared only about 23% of the businesses named, and another found the same short prompt retrieved 14 URLs in one run and 40 in another. One screenshot is an observation, not your visibility score."}},{"@type":"Question","name":"How do I calculate mention rate and citation rate in AI answers?","acceptedAnswer":{"@type":"Answer","text":"Run each of the 20 prompts five times per engine, then calculate mention rate as answers naming you divided by answers tested, and citation rate as answers citing your page divided by answers tested. For Google, also calculate citations among the answers where an AI Overview actually triggered, and keep the trigger count beside it."}},{"@type":"Question","name":"Which AI bots should I look for in my server logs?","acceptedAnswer":{"@type":"Answer","text":"Filter your logs for ChatGPT-User, OAI-SearchBot, GPTBot, PerplexityBot and Perplexity-User. Do not lump them together: OAI-SearchBot surfaces sites in ChatGPT search, GPTBot is for training, ChatGPT-User handles user-triggered fetches, and PerplexityBot surfaces sites in results while Perplexity-User fetches for a user question and generally ignores robots.txt."}},{"@type":"Question","name":"What is the 30-day schedule for testing one AI fix?","acceptedAnswer":{"@type":"Answer","text":"Week 1 is the baseline: run all 20 prompts five times in each of the three engines and pull 30 days of bot logs. Weeks 2 and 3, make one documented change. Week 4, retest with the same prompts and controls, then compare answer rates and bot requests over matched log windows."}},{"@type":"Question","name":"How many repeats do I need before treating a result as a real pattern?","acceptedAnswer":{"@type":"Answer","text":"Treat one appearance in five runs as a reason to keep watching, two or three as an emerging pattern, and four or five as something to retest. After a fix, a citation rate move from 10% to 15% is not a win on its own, so check the individual rows and repeat next month."}},{"@type":"Question","name":"What does it mean if bot requests rise but citations stay flat?","acceptedAnswer":{"@type":"Answer","text":"The bot reached your site but the page still did not appear as a source in the answers you sampled. Inspect the questions and pages being selected before changing access rules again. The reverse, citations with flat requests, means you should not claim a fresh fetch caused them."}}]}]}
```