An AI visibility report can be accurate and still fail to show that a provider helped a single new buyer find you. I see the same trick repeatedly: count a bot hit, a citation, and a screenshot where ChatGPT named you, then roll crawler access, source use, brand recommendations, and revenue into one cheerful score. The numbers can be real. The implied result is another matter.

Myth 1: “More AI bot hits mean more buyer visibility”

I used to tell clients bot traffic was a good sign. I was wrong, or at least sloppy. A log file full of GPTBot and PerplexityBot hits feels like demand, but crawling is not shopping.

Start with verification. User-agent strings prove nothing until you check requests against the relevant published IP lists. OpenAI publishes searchbot.json, gptbot.json, and chatgpt-user.json; Perplexity publishes its own bot files. Unverified requests do not belong in a verified AI-traffic total. A convincing name in a log is not identification.

Then separate what the verified bots actually do:

  • Background crawling: collecting pages, not demonstrating a buyer saw them.
  • Search indexing: crawlers such as OAI-SearchBot building the index.
  • Live-answer retrieval: user-triggered fetchers such as ChatGPT-User opening a page to help answer a question.

That last bucket gets you closer to an answer, not automatically into it. Retrieval still does not prove your brand was named or recommended. I wrote up what 513 live-answer fetches actually prove, and what they don’t, because access is not endorsement.

Require the traffic split in writing. Exclude unverified requests from verified totals, separate background crawling from live-answer fetches, and keep answer mentions separate again. Anything blended is a vanity total wearing a technical hat.

This one survives because the log looks so convincing. ChatGPT-User fetched your pricing page at 14:32, so the answer at 14:32 must have recommended you. I get the appeal. I have stared at those timestamps myself.

But a user-triggered fetch only proves the model opened the page while building an answer. It says nothing about the wording it chose after reading other pages beside yours. Retrieval shows access for a live answer, not that your brand was named in it.

The model can read your pricing page and recommend somebody else. The server log will still look lovely.

What is true instead: the fetch is one piece of evidence; the answer text is another. An agency cannot substitute the first for the second just because the first is easier to collect.

No saved answer, no demonstrated recommendation. Ask for the answer text behind each claimed recommendation, with the conditions under which it was captured. If a fetch cannot be connected to an observed answer, report it as a fetch. There is no shame in that category until somebody renames it a win.

Myth 3: “A source citation and a supplier recommendation are the same win”

Being footnoted is not being picked. Your page can supply a fact the model quotes while a third party supplies the recommendation. That is the gap a citation count hides.

In one sample, 62% of citations resulted in no brand mention at all, and only 13.2% of brands were both cited and mentioned. For Google AI Overviews, brand sources were cited in 87% of responses, but a brand was actually named in only 20.7%.

Free tool: the AI Visibility Checker asks ChatGPT, Gemini, Claude, Perplexity and Grok the question your customers ask and shows whether they recommend you.

A citation can therefore establish source use without establishing that a buyer was told to consider your business. Those are different outcomes, however neatly they fit into the same slide.

The flip side matters too. The same analysis found that at least 92.7% of ChatGPT-recommended brands appeared in cited content, while brands appearing in six or more cited URLs were six times more likely to be recommended. That does not turn every citation into a recommendation. It makes citations raw material rather than the finished result.

Require three separate counts:

  • Times your pages were cited as sources.
  • Times your brand was named.
  • Times your business was shortlisted as a supplier to consider.

If an agency reports only the first, it is reporting ingredients and calling it a booked job. Ask which result its work is supposed to produce.

Myth 4: “One screenshot establishes a visibility gain”

A screenshot is an anecdote with better lighting. Run the same prompt twice and you often get different answers. That is why sampling each prompt three to five times per engine in logged-out sessions and reporting prompt-level presence rate is a starting point for measurement, not unnecessary fussing.

One run proves the model could name you once, under those conditions, on that day. It does not establish a trend.

The variance is not small. A SparkToro and Gumshoe test with about 600 volunteers and 2,961 runs found less than a 1-in-100 chance of getting the same brand list twice for ChatGPT and Google AI. Getting the same list in the same order was less likely than 1 in 1,000. That is why visibility percentage across many prompts is reasonable while ranking position is unreliable.

If the list keeps changing, a movement from third to second can be noise dressed as progress. Repeated presence across the same questions is a more useful thing to track.

Require a fixed prompt panel, repeated observations, and recorded conditions: date, engine, logged-out state, and location. Keep the denominator beside the percentage so you can see what was actually sampled.

No denominator, no trend. Just a screenshot somebody liked enough to save.

Myth 5: “Branded prompts prove new buyers can discover you”

Ask ChatGPT about your company and you have already done the discovery work for it. You supplied the name. An answer to that question may tell you something useful about how the brand is described, but it cannot show that an unfamiliar buyer would find it.

Branded prompts inflate scores and must be excluded from the discovery headline. In one practitioner test, about one in five prompts in an auto-generated panel was branded. That slice produced the entire headline score; the non-branded answers did not mention the brand at all.

Blend the two and the discovery gap disappears under reputation points. What the deck calls share of voice, I call two numbers stapled together.

The fix is boring and non-negotiable:

  • Keep most of the panel focused on unbranded category questions: the best X and how to solve Y kind.
  • Report branded questions separately, including what the answers say about the business.
  • Make non-branded presence the headline when the claim is improved buyer discovery.

The distinction matters because non-branded discovery and branded sentiment answer different questions. One asks whether a buyer can encounter you without knowing your name. The other starts with your name already in the room.

Require those numbers unstapled. If an agency will not show the split, its headline is not enough to judge discovery.

Myth 6: “A higher mention rate proves the provider caused it”

Say mention rate goes from 12% to 31% after a provider starts work. The arithmetic can be correct while the sales claim remains unproven.

AI answers shift when you publish, when competitors publish, and when the model itself changes. Demand seasonality also changes the questions buyers ask. A before-and-after chart does not sort those explanations for you.

This is where an accurate visibility report quietly turns into a claim of credit. The provider points to the date it started, points to the rising line, and leaves you to connect them. Convenient arrangement.

To judge the work, I want:

  1. A baseline: the starting presence rate on the same fixed prompt panel.
  2. A dated change log: pages published or changed, plus PR and review pushes during the period.
  3. A comparison set: observations that did not receive the same treatment, with the limits of that comparison made clear.

These do not make every attribution question disappear. They give you something to examine beyond coincidence. Without them, you cannot separate a provider doing good work from a provider standing near good timing.

A rising line is evidence of change, not automatic evidence of cause. A provider who will not show the baseline is asking you to pay for weather.

Myth 7: “Visibility gains establish attributable revenue”

This is the hardest one to kill because it feels like the happy ending. Mentions went up, so revenue must follow. I have watched that logic waste budget.

The problem starts with treating every visit as equally valuable. Different assistants can send different mixes of visitors, and those visitors do not necessarily convert at the same rate. In a 200-site Stripe cohort, Perplexity traffic generated $1.42 per visit against $0.87 for ChatGPT. Claude generated $1.94 per visit in B2B SaaS on only 6% of the volume, while AI traffic as a group converted at 2.7% against 1.4% for Google organic.

Those are cohort results, not a promise for your account. The useful point is that traffic volume and commercial value can rank differently. Chasing session count can invert the value ranking.

Attribution adds another problem. The same AI revenue attribution analysis describes AI-referred sessions being recorded as Direct when referrer information is stripped. So a visible referral total is not necessarily the complete picture. But missing attribution does not give an agency permission to claim every unexplained sale either.

I want the unglamorous join: identifiable AI referrals connected to qualified demand and payment records, with attribution gaps left visible. A mention is a mention. A visit is a visit. Neither becomes revenue because the report places it near a revenue chart.

Bar chart comparing traffic volume with revenue per visit

What I would require before hiring a provider

Say you are spending $20k a month and an agency sends a four-page PDF with a green arrow and the words visibility up 41%. I would send it back.

What I ask for instead fits on one page:

  • A sample report from a real account, with names removed.
  • The fixed prompt panel, branded versus non-branded split, runs per prompt, and exact dates.
  • Presence rate with its denominator, plus saved answers behind claimed recommendations.
  • Verified server logs separating background crawling from live-answer fetches.
  • Separate counts for citations, named mentions, and supplier shortlists.
  • A dated change log and the baseline used to judge progress.
  • Referral visits connected to qualified demand and revenue where attribution supports it, with misses and gaps included.

That last line matters most. A provider should not hide a broken commercial chain inside a healthy visibility score.

A combined SEO and AI search service should apply the same commercial discipline to both channels. For SEO, I want crawled and indexed pages, queries driving qualified clicks, and pipeline tied to those pages. For AI search, I want non-branded presence, the citation-to-recommendation split, verified live-answer fetches, and referrals connected to qualified demand.

That is how groas runs it when I have seen the work up close: one team owning paid ads, SEO, and AI answers together, changes logged with reasoning, and a named strategist setting guardrails. No dashboards to interpret. No percentage-of-spend fee that grows when your CPC grows. The point is accountable execution, not two reports sold as one service.

Ask for the commercial chain before you hire. Have the provider show an account where a non-branded presence gain led to referral visits and then qualified pipeline, with the misses left in. That is the standard I would bring to a conversation with groas, not a requirement to waive because the pitch sounds good.

Ask who owns the miss

Of the seven, this is the myth I expect to keep arguing about. It flatters both sides. The agency gets to report a lift without touching tracking, and the client gets to believe discovery is working without asking whether any qualified deal came from it.

I have sat in that meeting. The slide says mentions tripled. Nobody asks what closed. Three months later the CFO asks, and by then the budget is gone.

So I ask one more question on the sales call: What happens when presence rises and pipeline does not—and who writes that down where I can see it?

Weak providers change the subject to tracking more prompts. Good ones walk back through the attribution, the offer, and whether the pages that earned citations deserved a shortlist spot in the first place.

I would rather hire the team that shows me one flat month with a clear explanation than three up-months with no denominators. That honesty is the signal I trust when someone promises to get my business recommended in ChatGPT answers. A report that cannot connect the answer to revenue is not proof of demand. It is proof of typing.