---
title: "Why AI Visibility Scores Disagree—and What Your SEO Provider Should Report"
description: "An AI visibility score measures a sample, not a universal ranking. Learn what a reproducible report should show—and how to connect it to buyer discovery."
url: "https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca1881ee-5d24-4361-8e45-93c56323f226.png"
published: "2026-11-09T06:22:11.904Z"
modified: "2026-11-10T21:25:43.493Z"
---

November 9, 2026 · 10 min read

# Why AI Visibility Scores Disagree—and What Your SEO Provider Should Report

[](https://groas.com/author/alexander-perelman)

[Alexander Perelman](https://groas.com/author/alexander-perelman)Personally Tested

Head Of Product @ groas

[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

Updated November 10, 2026

![Two identical measuring cylinders hold the same level of blue water, yet differently spaced scales read 71 and 43, showing scores depend on how you measure.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca1881ee-5d24-4361-8e45-93c56323f226.png)

In this article

1. [Start with the denominator, not the percentage](#start-with-the-denominator-not-the-percentage)
2. [Keep the engines separate before combining them](#keep-the-engines-separate-before-combining-them)
3. [Repeat the question: one answer is not a standing position](#repeat-the-question-one-answer-is-not-a-standing-position)
4. [Separate being cited from being chosen](#separate-being-cited-from-being-chosen)
5. [Separate what the provider does from what the engine decides](#separate-what-the-provider-does-from-what-the-engine-decides)
6. [Build a report whose sample can be repeated](#build-a-report-whose-sample-can-be-repeated)
7. [Buy execution and accountable measurement](#buy-execution-and-accountable-measurement)

An AI visibility score measures a sample, not your place in a universal ranking. Two providers can report 71% and 43% for the same brand in the same week without either inventing a number.

The difference can start with something painfully ordinary: they asked different questions. As this breakdown of [why AI visibility scores disagree](https://www.geoptimizer.io/blog/why-ai-visibility-tools-give-different-scores) explains, changing the sample changes the result. Calling both results “visibility” does not make them comparable.

**Free tool:** the [AI Visibility Checker](https://groas.com/free-tools/ai-visibility-checker) asks ChatGPT, Gemini, Claude, Perplexity and Grok the question your customers ask and shows whether they recommend you.

**The score describes how often you appeared in these answers to these questions, on these engines, across these checks.** Change any of those inputs and you change what the percentage means.

I’ll use one example throughout: say you run a 40-person IT services firm in Austin. One question that matters to your business is “best managed IT provider for small law firms.” You want to know whether prospective buyers discover your firm when they ask it, not whether a dashboard can produce an encouraging number.

That distinction decides what your provider should measure, what it can reasonably promise, and what you should pay for.

## Start with the denominator, not the percentage

In a simple appearance-rate report, a provider selects buyer questions, runs them, and divides the observations in which you appeared by the total observations. The denominator is the set of opportunities you had to appear.

For our Austin firm, two providers might build very different tests:

- **Vendor A:** 12 prompts, mostly branded and local, such as “reviews of \[your firm name] for law offices.”
- **Vendor B:** 40 prompts, mostly non-branded and comparative, such as “best managed IT provider for small law firms” or “IT support cost per user Austin.”

Vendor A could reach 71% because its test leans toward questions you were already likely to win. Vendor B could reach 43% because its test includes harder questions where competitors and directories also qualify.

Neither percentage tells you which test better represents your buyers until you see the questions.

The formula matters, too. One [scoring example](https://crawltide.com/blog/ai-visibility-score) gives the same 20 answers a score of 31 with competitor weighting and 25 without it. That is a 6-point swing from arithmetic, not a change in the answers.

**Ask what counts as an appearance, which questions count, and how the provider combines them.** A percentage without that explanation is a number looking for a job.

## Keep the engines separate before combining them

The next layer is where those questions get asked. [ChatGPT, Gemini and Perplexity use different retrieval approaches and favor different sources](https://www.mukodigital.com/blog/why-chatgpt-gemini-perplexity-recommend-different-brands). The same buyer question does not make their answers interchangeable.

If Vendor A mostly checks Perplexity and Vendor B mostly checks ChatGPT, they are measuring different answer systems. An overall percentage can hide that difference, especially if the engine mix changes between reports.

For our Austin firm, I want to see whether the law-firm question produces a mention in each engine before anyone rolls the results into one score. Otherwise, an apparent improvement could simply mean the provider ran more checks on the engine where the firm already performs well.

**Keep the prompt list, engine mix, and run count stable when comparing periods.** If the sample changes, label the change rather than presenting it as growth.

## Repeat the question: one answer is not a standing position

Even with the same prompt and engine, the answer can vary. Our Austin firm might appear on Monday, disappear on Tuesday, and sit beside a competitor on Wednesday without anyone changing its website.

That makes repetition part of the measurement, not an optional extra.

The [sampling math](https://www.geoptimizer.io/blog/why-ai-visibility-tools-give-different-scores) illustrates the problem: five runs can leave a margin around ±43.8 percentage points, while 100 runs narrow it to about ±9.8 points at 95% confidence. A handful of checks should not carry the authority of a precise ranking.

Citation churn adds another reason to look beyond one answer. A [20-day citation study](https://ahrefs.com/blog/how-stable-are-ai-citations/) found substantial turnover by day two, with about 75% of lost citations returning within 20 days. A missing citation in one check does not necessarily mean a lasting loss.

I think of a screenshot as weather: it records what happened at that moment. The analogy stops there; repeated checks still describe the tested sample, not every answer every buyer receives.

**Use repeated observations to judge a pattern. Use screenshots to document individual answers.** They do different jobs.

## Separate being cited from being chosen

Now return to the buyer asking about managed IT for a small law firm. Several useful outcomes can occur:

- A **citation** links to your site as evidence.
- A **mention** names your firm, with or without a link.
- A **recommendation** presents your firm as a choice for the buyer.

These outcomes can overlap, but they are not interchangeable. This explanation of [citations, mentions and recommendations](https://www.clickradius.io/institute/citation-vs-mention-vs-recommendation.html) makes the distinction useful for reporting: your page can supply the answer without your firm becoming the recommended provider.

A clear definition or useful statistic may make your page worth citing. A competitor with stronger reviews and a bar association directory listing may still become the recommendation. Your content and the evidence around your business are doing different work.

**Being the source is not the same as being the choice.** Count both, but do not let one stand in for the other.

Neither outcome proves that a buyer visited, booked a consultation, or became a customer. The [traffic comparison](https://www.kampaignlab.com/seo/search-beyond-google-where-perplexity-chatgpt-and-bing-traffic-actually-lands) cited in the draft puts AI referrals at about 0.32% of traffic across 101,574 sites in 2026, against 42.75% for organic search. Visibility and visits are plainly different measures.

For our Austin firm, the reporting chain needs to stay visible: the answer names the firm, a buyer discovers it, and that discovery may lead to a consultation. A citation no buyer clicks is not a visit. A recommendation with no attributable pipeline is not a sale.

Count the observations separately before connecting them to business outcomes.

## Separate what the provider does from what the engine decides

Once those outcomes are clear, the scope of the work becomes easier to judge. For our Austin firm, I would organize it around three inputs:

1. **Pages that answer the buyer’s question.** A service page can explain the offer in plain language, with a price range per user, a response-time promise, and a clear definition the model can use.
2. **Technical access.** Crawl blocks, script-dependent content, and answers buried in video can prevent useful material from being accessible.
3. **Corroboration beyond the website.** Reviews, relevant listings, and detailed law-firm case studies help support the claims made on the service page.

The mechanism matters: accessible, useful pages give an engine material to draw from; corroborating evidence helps support the business as an option. That is work a provider can undertake. It is not control over the answer.

No agency decides whether ChatGPT or Gemini selects your firm on a particular day. Models change, retrieval changes, and competitors publish. A [buyer checklist for AI visibility agencies](https://www.5wpr.com/new/ai-visibility-agency-questions/) is useful precisely because it separates deliverables from promises about results the supplier cannot control.

**The provider owns the work and its measurement. The engines own the answer.** Put that boundary in writing before money moves.

## Build a report whose sample can be repeated

For our Austin firm, I would start with 25 to 30 buyer questions, split into branded, local non-branded, and comparison prompts. The [30-prompt test](https://groas.com/post/the-30-prompt-test-measure-your-brand-s) provides a starting shape.

The list stays stable month to month. Any addition or removal gets dated in the report. Otherwise, the provider can change the exam while showing you the grade.

A reproducible report needs to disclose:

- The full prompt list, including exact wording.
- The engines checked, with per-engine results, including Copilot where it is part of the agreed scope.
- The checking schedule and runs per prompt.
- Separate counts for citations, mentions, and recommendations.
- Separate branded and non-branded results.
- Model versions where recorded, plus dated answer examples.
- Any change to the sample or scoring formula.

**If the provider will not show the denominator, you cannot judge the percentage.** That is not a technical objection. It is the minimum needed to understand what you bought.

Weekly checks turn the list into a record. Four checks of the same prompt in a week offer more context than one screenshot, but they still do not make a precise estimate. Keep the run count visible rather than letting a tidy chart imply certainty.

For the monthly readout, I like the sequence in this [reporting structure](https://searchscore.io/guides/geo-reporting-for-clients/): business context, current observations, changes, dated evidence, referral trends, and next actions. Start with the business question. End with what gets done next.

### Keep three ledgers: work, visibility, and business

A report that mixes implementation, sightings, and sales into one percentage makes the person defending the spend do detective work. I want three _ledgers_ for our Austin firm:

| Ledger              | What belongs in it                                                                    |
| ------------------- | ------------------------------------------------------------------------------------- |
| Work shipped        | Pages published, crawl blocks fixed, reviews and listings updated, with dates         |
| Visibility observed | Citations, mentions, and recommendations across the fixed prompt set, split by engine |
| Business outcomes   | AI referrals, assisted calls, and booked consultations                                |

The implementation record can be concrete: a law-firm IT comparison page published February 4, a pricing crawl block fixed February 6, and two client reviews added February 9. These are illustrative entries, not evidence of a result.

The next ledger records what the repeated checks actually show. The final one records whether buyer activity follows. The same [reporting guidance](https://searchscore.io/guides/geo-reporting-for-clients/) warns against score movements presented without implementation records or supported only by single-check screenshots.

**A score change beside a work log is something to investigate, not automatic proof that the work caused it.** Without the work log, you have even less to go on.

For the recurring questions, I keep a plain-English explanation in [_AI Visibility Is Not a Screenshot_](https://groas.com/post/your-ai-visibility-measurement-questions). The working rule is simple: repeat the checks, keep the categories separate, and look for a sequence you can inspect. The page ships. Visibility for that prompt family may become steadier. Referrals follow, or they do not.

That last possibility belongs in the report, too.

## Buy execution and accountable measurement

If I had to defend this spend to a finance lead, I would bring the prompt list, engines, run counts, and the three ledgers. Not just the headline score.

The purchase is straightforward. The provider builds buyer-focused pages, keeps them accessible, works on corroborating evidence, and measures the agreed sample consistently. The business then has a record of what changed and whether discovery or consultations followed.

**Pay for executed work you can audit and counting you can repeat.** Do not pay for the emotional comfort of a percentage whose denominator keeps moving.

### Put the measurement rules in the contract

The statement of work should name the pages to build or fix, technical access checks, and review and listing work, with dates. Its reporting section should specify:

- The prompt list and engine mix.
- Runs per prompt per week.
- Separate citation, mention, and recommendation counts.
- How changes to the sample get recorded.
- Which referral and consultation measures accompany the visibility report.

Anything promising our Austin firm a specific ChatGPT citation by March gets crossed out. No supplier controls that choice.

**Contract for actions and transparent observation, not guaranteed mentions.** A promise that ignores the engine’s independence does not become more credible because it appears in a proposal.

### Where groas fits—and how to judge the next report

This is where I put groas for the Austin buyer. groas combines done-for-you paid and organic search work with logged actions and human strategic ownership. Its SEO and AI search work follows that operating model: page work, technical access, corroboration, and reporting on what the sample shows.

If you are comparing providers, use the [ranked shortlist of ongoing SEO and GEO services](https://groas.com/post/ongoing-seo-and-geo-services-a-shortlist-for-chatgpt-gemini), then judge the measurement behind the pitch. The useful distinction is managed execution with an inspectable record, not another dashboard you have to interpret yourself.

**The groas case rests on accountable work and transparent measurement, not a guaranteed mention rate.** For our Austin firm, the practical requirement is 30 buyer questions, checked weekly, tied to shipped work and a referral line the finance lead can inspect.

Now apply that model to the next report. Suppose the firm’s score moves from 43% to 71%, but the provider has replaced difficult comparison questions with branded ones. Start with the denominator: the sample changed. You cannot call that a comparable improvement. Ask for the original prompt set to be rerun and the new questions to be reported separately.

If the sample stays fixed instead, inspect the per-engine observations, then the work log, then referrals and booked consultations. The higher score may reflect more frequent visibility. Whether it improves buyer discovery is the next question, not an assumption.

That is the standard I would hold groas to. Keep the sample reproducible and the work visible, and the 71% versus 43% argument ends.

## Frequently Asked Questions

### Why do two AI visibility tools give completely different scores for the same brand?

Because each tool measures its own sample: the prompts it runs, the engines it checks, and how it counts appearances. One tool asking mostly branded, easy questions can report 71% while another asking harder comparative questions reports 43%. Calling both results visibility does not make them comparable.

### What should I ask an AI visibility provider before trusting the percentage they report?

Ask what counts as an appearance, which buyer questions were included in the test, and how the provider combines the observations into one number. Scoring choices alone can shift a result, such as the same 20 answers scoring 31 with competitor weighting and 25 without. A percentage without that explanation is a number looking for a job.

### Do ChatGPT, Gemini and Perplexity give the same answers about which brand to pick?

No. They use different retrieval approaches and favor different sources, so the same buyer question can produce different answers on each engine. An overall percentage that mixes engines can hide this, so visibility results should be reported per engine before anyone rolls them into one score.

### How many times should a prompt be checked before an AI visibility result means anything?

A prompt should be checked repeatedly, because one answer can vary by the day even with no change to the website. Five runs can leave a margin around plus or minus 43.8 percentage points, while 100 runs narrow it to about plus or minus 9.8 points at 95% confidence. Use repeated observations to judge a pattern and screenshots only to document individual answers.

### What is the difference between a citation, a mention and a recommendation in AI answers?

A citation links to your site as evidence, a mention names your firm with or without a link, and a recommendation presents your firm as a choice for the buyer. They are not interchangeable: your page can supply the answer without your firm becoming the recommended option. Count all three separately, and none of them proves a buyer visited, booked, or became a customer.

### Can an AI visibility agency guarantee my brand gets cited by ChatGPT?

No. A provider can build pages that answer buyer questions, keep content technically accessible, and strengthen corroborating evidence like reviews and listings, but the engines decide what appears in any given answer. Models and retrieval change and competitors publish, so contract for executed work and transparent measurement rather than guaranteed mentions.

### What should a reproducible AI visibility report include?

It should disclose the full prompt list with exact wording, the engines checked with per-engine results, the checking schedule and runs per prompt, and separate counts for citations, mentions and recommendations. It should also show branded and non-branded results separately, model versions where recorded, dated answer examples, and any change to the sample or scoring formula. If the provider will not show the denominator, you cannot judge the percentage.

### How should I connect AI visibility reporting to actual business results?

Keep three ledgers: a work log of pages published, crawl blocks fixed and listings updated with dates; a visibility record of citations, mentions and recommendations across a fixed prompt set split by engine; and a business ledger of AI referrals, assisted calls and booked consultations. A score change beside a work log is something to investigate, not automatic proof that the work caused it.

## Run Your Marketing On groas

Plug ChatGPT, Claude or any AI agent into groas and let it do the work of a whole marketing team: free to ask, $49 a month flat with every feature included, and autopilot whenever you’re ready.

[Start Free](https://groas.com/pricing)

[Or have groas run it for you](https://groas.com/done-for-you)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","description":"groas is the marketing infrastructure for your own AI: plug Claude, ChatGPT or any AI agent into groas and run all your marketing from it, for a tiny fraction of the usual cost, with the groas app there whenever you want to see more. It is the layer ChatGPT, Claude and any AI assistant run a business’s online marketing on. Through groas an assistant can do anything you could do natively in Google Ads, Bing Ads, Meta Ads and ChatGPT ads, on your website and in AI search, and, where the owner switches it on, can run connected campaigns by itself, so AI agents do the work a whole marketing team used to (free to read, $49 a month to make changes, delivered as an MCP server). Separately, its done-for-you service has senior people and proprietary quantitative systems run paid ads, SEO and AI search for businesses, and white label for agencies, from $5,000 a month.","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#webpage","url":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency","name":"Why AI Visibility Scores Disagree—and What Your SEO Provider Should Report","description":"An AI visibility score measures a sample, not a universal ranking. Learn what a reproducible report should show—and how to connect it to buyer discovery.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca1881ee-5d24-4361-8e45-93c56323f226.png"},"breadcrumb":{"@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#breadcrumb"},"dateModified":"2026-11-10T21:25:43.493Z","mainEntity":{"@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Why AI Visibility Scores Disagree—and What Your SEO Provider Should Report","item":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#article","url":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency","mainEntityOfPage":{"@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Why AI Visibility Scores Disagree—and What Your SEO Provider Should Report","description":"An AI visibility score measures a sample, not a universal ranking. Learn what a reproducible report should show—and how to connect it to buyer discovery.","datePublished":"2026-11-09T06:22:11.904Z","dateModified":"2026-11-10T21:25:43.493Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca1881ee-5d24-4361-8e45-93c56323f226.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"provider, visibility, report, firm, buyer, answer, work, visibility scores","wordCount":2149,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"My background includes Goldman Sachs and Stanford Computer Science.","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/why-ai-visibility-reports-disagree-and-what-your-seo-agency#faq","mainEntity":[{"@type":"Question","name":"Why do two AI visibility tools give completely different scores for the same brand?","acceptedAnswer":{"@type":"Answer","text":"Because each tool measures its own sample: the prompts it runs, the engines it checks, and how it counts appearances. One tool asking mostly branded, easy questions can report 71% while another asking harder comparative questions reports 43%. Calling both results visibility does not make them comparable."}},{"@type":"Question","name":"What should I ask an AI visibility provider before trusting the percentage they report?","acceptedAnswer":{"@type":"Answer","text":"Ask what counts as an appearance, which buyer questions were included in the test, and how the provider combines the observations into one number. Scoring choices alone can shift a result, such as the same 20 answers scoring 31 with competitor weighting and 25 without. A percentage without that explanation is a number looking for a job."}},{"@type":"Question","name":"Do ChatGPT, Gemini and Perplexity give the same answers about which brand to pick?","acceptedAnswer":{"@type":"Answer","text":"No. They use different retrieval approaches and favor different sources, so the same buyer question can produce different answers on each engine. An overall percentage that mixes engines can hide this, so visibility results should be reported per engine before anyone rolls them into one score."}},{"@type":"Question","name":"How many times should a prompt be checked before an AI visibility result means anything?","acceptedAnswer":{"@type":"Answer","text":"A prompt should be checked repeatedly, because one answer can vary by the day even with no change to the website. Five runs can leave a margin around plus or minus 43.8 percentage points, while 100 runs narrow it to about plus or minus 9.8 points at 95% confidence. Use repeated observations to judge a pattern and screenshots only to document individual answers."}},{"@type":"Question","name":"What is the difference between a citation, a mention and a recommendation in AI answers?","acceptedAnswer":{"@type":"Answer","text":"A citation links to your site as evidence, a mention names your firm with or without a link, and a recommendation presents your firm as a choice for the buyer. They are not interchangeable: your page can supply the answer without your firm becoming the recommended option. Count all three separately, and none of them proves a buyer visited, booked, or became a customer."}},{"@type":"Question","name":"Can an AI visibility agency guarantee my brand gets cited by ChatGPT?","acceptedAnswer":{"@type":"Answer","text":"No. A provider can build pages that answer buyer questions, keep content technically accessible, and strengthen corroborating evidence like reviews and listings, but the engines decide what appears in any given answer. Models and retrieval change and competitors publish, so contract for executed work and transparent measurement rather than guaranteed mentions."}},{"@type":"Question","name":"What should a reproducible AI visibility report include?","acceptedAnswer":{"@type":"Answer","text":"It should disclose the full prompt list with exact wording, the engines checked with per-engine results, the checking schedule and runs per prompt, and separate counts for citations, mentions and recommendations. It should also show branded and non-branded results separately, model versions where recorded, dated answer examples, and any change to the sample or scoring formula. If the provider will not show the denominator, you cannot judge the percentage."}},{"@type":"Question","name":"How should I connect AI visibility reporting to actual business results?","acceptedAnswer":{"@type":"Answer","text":"Keep three ledgers: a work log of pages published, crawl blocks fixed and listings updated with dates; a visibility record of citations, mentions and recommendations across a fixed prompt set split by engine; and a business ledger of AI referrals, assisted calls and booked consultations. A score change beside a work log is something to investigate, not automatic proof that the work caused it."}}]}]}
```
