A green arrow is not enough evidence to put search work on the roadmap. Before I trust an AI visibility dashboard, I want to see whether its records separate a repeatable buyer-question gap from changing answers and measurement blind spots.
This is a second look at By 2028, One AI Visibility Score Won’t Win an SEO Contract, for a decision that cannot wait until 2028: whether the tool you already pay for helps you make better decisions. This two-week audit checks what sits behind its numbers—and whether the evidence justifies your next move.
The question is narrower than “is the score accurate?” Inspectability is the test: can you trace a reported gap back to observable answers? If the dashboard says you are missing from buyer questions on Perplexity, can you open the exact prompt, date, answer text and cited URLs?
My expectation is that many tools will expose at least one blind spot. The mechanism is straightforward: vendor-selected prompts, API or logged-out collection, and pooled scores can hide different things. Without the underlying records, you cannot tell a repeatable buyer gap from weather.
Define the three deliverables before you start
After two weeks, you should hold:
- An evidence sheet with one row per question, engine and round.
- A written list of measurement limits.
- One next action with supporting evidence, or a documented reason not to act.
A reason to hold is a result, not a failed audit. A panel that surfaces no repeatable gap gives you no reason to chase one because a dashboard turned weather into a trend line.
This protocol tests the instrument. Whether a content fix lifts your mention rate is a separate experiment with its own control. Mixing the two is how a monitoring report quietly becomes a pitch deck.
Fix a panel of 12 buyer questions
Start with questions buyers already ask: sales-call transcripts, support tickets, the “how do you compare to X” thread your team answers every week. That is evidence of demand. Label any invented question as a hypothesis, because a question nobody asks can surface a gap that costs you nothing.
Choose 12 questions you can keep running for two weeks. Record why each earns its slot, then sort them into three types:
| Question type | Example wording | What absence could mean |
|---|---|---|
| Category discovery | “best [category] for [use case]” | You are absent before a shortlist exists |
| Comparison | “X vs Y,” “alternatives to X” | The shortlist forms without you |
| Constraint-specific | “[product] under $2,000,” “[service] for regulated industries” | Buyers with your exact need do not see you |
The mix matters. In a 22,295-answer B2B study, prompt archetype alone shifted mention rate by 16.7 percentage points across engines (Lifewood’s breakdown). Two tools with different prompt mixes can report different numbers while describing their own panels honestly.
Write down the panel before collection. Do not improve the questions halfway through because the answers disappoint you.
Lock the conditions before collecting answers
Uncontrolled conditions can mislead you without anyone behaving badly. Record how each answer was collected, not just what it said:
- Exact prompt wording, language and market.
- Account context, including whether you are logged in.
- Interface or API.
- Model, where disclosed.
- Search mode and whether web search ran, where observable.
That last distinction matters. ChatGPT ran web search on only 34.5% of US queries as of February 2026 (Semrush data, via Magnitude’s evaluation questions). A model answering from memory and the same model searching the web are different instruments wearing one product name.
Run every question in a fresh conversation so answer one cannot seed answer two. Keep settings consistent where the product allows it. Record what you cannot control as a limitation, not an excuse to skip the observation.
The viewer matters too. Practitioners note that trackers usually query APIs or logged-out sessions, while buyers may arrive logged in, with memory and location active (the r/GEO_optimization discussion). If your dashboard collects differently from your manual panel, that difference belongs in the sheet from round one.
Run three rounds: 108 planned observations
Run all 12 questions on ChatGPT, Gemini and Perplexity in each of three rounds spread across two weeks. That gives you 108 planned observations.
Keep the panel and collection conditions fixed across rounds. Keep your site still where practical, too. Log unavoidable changes, such as a new landing page or product launch, so you can account for them when reading the results.
This sample cannot estimate market-wide visibility. Its job is narrower: to check whether the dashboard’s numbers trace back to observable answers and whether an apparent gap persists across repeated observations. Do not promote a small instrument check into a market-share claim.
Build one evidence row per question, engine and round
Your evidence sheet is the deliverable. Each row should hold:
- Question ID, exact wording, engine, timestamp and round.
- Collection conditions from the previous section.
- Full answer capture: pasted text, not a summary.
- Brand mention: yes or no.
- Linked citation and cited URL, if any.
- Recommendation: whether the answer picks your brand.
- Competitor presence.
- Collection status: collected, failed or blocked.
Store the raw answer. Summaries are where interpretations sneak in. For a starting structure, use the spreadsheet tracker from the groas post on measuring mention rates.
Three fields deserve particular care: brand mention, linked citation and recommendation are different outcomes. A practitioner logging by hand keeps those distinctions explicit: cited as a source, named in the text, or picked as the recommendation (the r/aeo discussion).
A tester reviewing 10+ AEO tools reached a similar practical conclusion: repeated cited pages and competitor appearances were more useful than the score alone (the hands-on review).
Keep those outcomes separate. A mention is not a linked citation, a citation does not establish a visit, and neither establishes a sale.
Request the dashboard’s records before round one
Before collecting your own panel, request four things from the vendor or find them in the tool’s settings:
- The prompt panel it runs.
- Its collection method and conditions.
- Its engine coverage.
- The underlying answer records.
You need the questions behind the score. As a builder explains in the same r/aeo discussion, trackers run selected questions on your behalf; they do not see the prompts real users type.
“Which prompts trigger competitors” therefore means “which tested questions returned competitors.” If the panel mostly names your brand, its numbers may flatter you. If it mostly uses category terms, it may look worse than your sales conversations suggest. Read the list before treating it as evidence about your buyers.
Where possible, match your observations to stored dashboard records. Compare the prompt, engine, timing and collection conditions. The same wording on the same engine at roughly the same hour is a candidate comparison, not automatic equivalence. Label material differences non-equivalent, not tool failure.
Disagreement alone is not a scandal. When five AI visibility tools scored the same domain, citation counts differed by as much as 8.2x, driven by sampling cadence and counting definitions (MR Research, via ITMaster). Ask how often the tool reruns each prompt and whether it counts unlinked mentions as citations.
Record every missing field as a limitation. An invisible prompt panel is not proof your brand was absent. It is proof you cannot verify the claim.
Classify repeatability without hiding engine differences
For each engine-question pair, classify brand presence across the three rounds:
| Bucket | What the records show |
|---|---|
| Repeated absence | Brand missing in all three rounds |
| Intermittent presence | Brand present in one or two rounds |
| Repeated presence | Brand present in all three rounds |
| Insufficient evidence | Failed or blocked collection prevents a complete three-round classification |
Report the observations you do have, including failures. Keep counts and denominators visible: “named in 2 of 3 rounds” and “cited in 0 of 3,” not “visibility improving.”
Do not pool away the gap you are trying to find. A 3-of-3 on Perplexity can hide a 0-of-3 on Gemini. In a study of 67,144 answers, 84% of sources cited for a question appeared on only one engine (GetMentions’ findings). Your gap may be engine-specific.
Repeated absence is the bucket I would use to nominate work in this audit, not to declare a proven cause. Answers wobble. In that same study, 69% of a typical answer’s sources changed overnight; Perplexity’s daily source churn was 44% and Gemini’s was 88%. One domain Hami Tahm tracked appeared in 68 of 68 Perplexity runs, then in 2 of 68 forty hours later (the measurement).
Consider a hypothetical row: question 7, comparison type, Gemini, brand named in 0 of 3 rounds, competitor X named in 3 of 3, with the competitor’s comparison page cited each time. That is worth investigating.
The same question on Perplexity, with your brand named in 2 of 3 rounds and different URLs cited each time, belongs in intermittent presence. It is evidence of variation, not enough evidence of a repeatable absence. Keep the classification honest.
Turn one supported gap into an accountable next step
Choose one repeated-absence pattern: the question closest to money, on an engine your buyers actually use. Treat it as a candidate for investigation, not an automatic content brief.
Work through the possible explanations before prescribing a fix:
- Retrieval may never reach your page. Use the fetch-pipeline breakdown to frame that investigation.
- Your page may be retrieved but offer nothing quotable.
- A competitor may have a comparison page you have never written.
Answer captures show what appeared and what got cited. They do not, by themselves, reveal whether an uncited page was fetched. Keep that distinction in the action record rather than turning a plausible explanation into a finding.
Then write down the proposed change, supporting question IDs and engine-round counts, the owner, and the conditions for follow-up measurement. If the evidence cannot support a specific change, record what you still need to establish.
Size the follow-up honestly. If a brand appears in 30% of answers, 20 answers put the true rate anywhere from roughly 15% to 52%. A move from 25% to 35% across two sets of 100 answers has overlapping intervals (the confidence-interval walkthrough). If the retest cannot detect the move you hope to see, it is a ritual, not a measurement.
Keep four records separate:
- What you shipped.
- What the answers did afterward.
- What referral traffic arrived.
- What revenue you can attribute.
A before-and-after change does not establish causation on its own. The fix test needs its own control; the separate citation-test protocol covers that half. This audit identifies a repeatable observed gap. It does not get to claim the cure.
Keep the tool, demand its records, or defer the work
The final question is practical: can the dashboard’s records support the decisions you need to make this quarter? Use the evidence sheet and limitations list to choose one of three outcomes.

Keep the tool when its stored records hold up against comparable observations: exact prompt wording, answer text you can open, cited URLs and denominators that survive your cross-check. Inspectable records make the dashboard useful even when individual answers vary.
Demand better evidence when the questions matter but the measurement cannot be inspected. Ask for the prompt panel, raw answer exports, run counts and collection conditions. If support cannot produce them within a week, record that as a finding. Before canceling, check whether those records sit behind an enterprise tier. A pricing decision is different from a product with nothing to show.
Defer the work when your panel is inconsistent or the question has little connection to revenue. A gap that appears once, on one engine, for a question your buyers never ask can wait. Dashboards manufacture urgency. Your roadmap does not have to accept it.
If 12 questions and three rounds exceed what you can commit to this quarter, the lighter mention audit uses a smaller panel. It still beats trusting the score.
The deliverable is not a higher visibility number. It is 108 planned evidence rows, a list of what you could not measure, and one action with an owner and retest conditions—or a documented reason to hold. A dashboard that passes earns a role in your decisions. One that cannot produce its records does not earn the right to direct your roadmap.
That is where measurement hands off to execution. groas makes changes inside the guardrails you set, logs every action with its reasoning, and puts one named strategist on the outcome. Your sheet identifies the gap worth acting on. Only execution closes it.

