LinkedIn
ChatGPT names your garage door company on Monday and leaves it out on Thursday. That change is not a verdict on your agency.
Say you run a shop in Austin. You ask: “Best garage door repair in Austin for same-day spring replacement.” On Monday, ChatGPT names you second and links your booking page. On Thursday, you ask the same question and disappear. No site change, no lost reviews, no competitor announcement in between. I have watched owners treat that second answer as proof that someone broke their visibility. It proves much less.
Everything here follows from one fact: an AI answer is generated fresh, not pulled from a fixed list. OpenAI documents its Chat Completions as non-deterministic by default. The same request can produce a different answer on another run. A changed shortlist does not, by itself, show that an agency helped you or hurt you.
To judge the service you are buying, keep three observations apart: did the engine name you, did it link to you, and did it put you forward for a buyer ready to book? I will use that Austin spring-replacement question throughout. The buyer’s words can stay still while the answers move.
The same question is not a fixed search result
Think of the Austin prompt as two layers, not one scoreboard. When browsing runs, the first layer fetches pages from search. The second writes an answer from what it has available, choosing its words and brands as it goes. If the fetched material changes, the answer has different material to work with. Even if it does not, the answer can still change.
That is where the analogy stops: these are not two tidy machines with a visible handoff you can inspect in every answer. But the distinction is useful. A tracking guide from Siftly notes that a single query may name your brand today and omit it tomorrow without a content change. It advises buyers to look for a pattern across repeats rather than treating one hit as a result.
Seer Interactive showed how much the search step can vary. It ran a byte-identical 148-character prompt twice, eight minutes apart, and the runs shared zero exact underlying search query strings. In a longer comparison from Sept 3 to Sept 16, retrieval went from 14 URLs to 40 for the same idea. The buyer’s question had not become a new question. The material gathered for the answer had changed.
I used to tell clients that a dropped mention meant a competitor had passed them. I was wrong. An omitted name is an observation, not a diagnosis. For the Austin question, log the prompt, engine, time, and whether web search ran before you try to explain the answer. Otherwise you cannot even tell what you are comparing.
A mention, a citation, and a recommendation are three different wins
A report that calls every appearance “visibility” skips the part you need to know. Here are the three counts I want separated:
- Mentioned: The answer says your business name.
- Cited: The answer links to a page on your site as a source.
- Recommended: The answer puts your business forward as an option for this buyer.
They can occur together, but they do not have to. A Semrush analysis with Kevin Indig across 3,981 domain appearances found 61.7% were ghost citations: sites cited but never named. Only 13.2% were both cited and named. Your page can help supply an answer while another shop gets the recommendation.
Run that through Austin. On Monday, ChatGPT says Apex Garage Door Co. does same-day springs in Austin and links your booking page. You were mentioned, cited, and recommended. On Thursday, it explains torsion springs and links your spring-price guide at the bottom, but names two other shops in the answer. You were cited. You were not mentioned or recommended. Calling both days a “visibility win” hides the difference a shop owner actually cares about.
I keep the Mentioned, Cited, Clicked glossary open when reviewing reports because the fix depends on what is missing. A linked guide shows that the page can enter the source set. It does not show that the business became a choice in the written answer. Ask for three columns, not one blended score.
One screenshot cannot tell you whether visibility improved
Before the engine writes a word, you have already made four choices: the prompt wording, the engine, the time, and whether browsing runs. Change “best” to “cheapest” in the Austin question and you have changed the buyer’s request. Ask Gemini instead of ChatGPT and you have changed engines. Ask at another time and a live search may bring back different pages. If you do not record whether web search ran, you may be comparing a fetched answer with one that did not fetch.
Prefer ran a useful public repeat test: 80 questions asked three times on four engines, for 960 answers. Repeat-run citation overlap was 37% for ChatGPT and 35% for Gemini, versus 96% for Perplexity and 68% for Claude. ChatGPT averaged 3.05 sources per answer across 218 sites; Perplexity averaged 19.48 across 748 sites. Those are results from that study design, not permanent reliability grades. They do show why one screenshot, or one blended rate across engines, throws away useful information.
A lower-overlap engine needs more repeats before you call an appearance a trend. An engine that cites more sources gives your site more chances to appear, but it can also put more competitors beside you. Neither pattern tells you what happened to bookings.
Repeated observation is less glamorous than a screenshot. Ask the same buyer question the same way, more than once, and count what happens. Siftly suggests running a query 2 to 3 times per 24 hours and treating 50% or better as stable, with 5 to 10 runs for more confidence and larger samples for share of voice. For Austin, run the spring-replacement question on ChatGPT across three mornings and record named, linked, picked each time. Report “named in 2 of 3 runs,” not “missing on Thursday.” The first statement describes the observations. The second just describes the screenshot someone chose.
If a report skips the per-engine counts, use the method in Your AI Visibility Score Is a Sample, Not a Ranking to rerun the questions yourself before paying for another month. A screenshot is a sample, not a before-and-after study.
Compare the same buyer questions in two windows
Now the comparison can earn its name. Say your Austin list contains 10 buyer questions about springs, openers, and emergency calls. You run each question three times on ChatGPT this week and record 12 mentions across 30 answers. A month later, you run the same 10 questions three times each and record 19 mentions across 30 answers.
That is a gap worth discussing because you held the basics still: same questions, same engine, same count method, two windows. I send owners to How to Measure AI Visibility Without Tracking Fake Rankings for the method of keeping the question set fixed while answers change.
Keep the other two columns beside mentions. Did links rise too? Did the shop appear as a recommendation more often, or did its guide merely turn up in more source lists? Report each engine on its own line as well. ChatGPT moving from 4 of 10 to 7 of 10 while Perplexity stays at 9 of 10 is not a contradiction. The engines fetch and write differently. Blend those rates and you conceal where the change happened.
Then keep frequency apart from money. A higher named rate means the shop appeared in more of the answers you measured; it does not mean more people booked. Even a clean rise does not prove the agency caused it. Retrieval can change during the month. Competitors can edit pages. Reviews can land. Your comparison shows that the shop appeared more often after the work. It cannot assign every extra appearance to that work.
That does not make measurement pointless. It makes the claim precise. Buy a repeatable comparison, not a promise that every changed answer has one cause.
Buy the work behind the answer, not the screenshot
Measurement does not put a garage door shop into an answer. When an engine searches, it needs useful material to retrieve before it can name a business in its response. A DigitalCave check from Oct 2026 states there is no form, feed, or paid listing that puts a business into ChatGPT, Perplexity, Copilot, or Gemini. So the Austin work is less mysterious than the pitch usually sounds: a page for each buyer job, with same-day springs, pricing, and booking made clear; technical access open to OAI-SearchBot and PerplexityBot; Bing Places claimed with matching name and hours; and fresh reviews that corroborate what the shop says it does. Tracking can show what happened afterward. It is not the implementation.
That is my proposal filter. If the deliverable is a screenshot and a score, I pass. If it includes pages, technical fixes, and work on the sites the engines pull from, I keep reading. groas offers SEO and AI Search for a flat $1,499 a month, covering Google as well as ChatGPT, Perplexity, Gemini, and other AI search surfaces, with weekly visibility reporting against the same buyer-question set. The point is not to claim a one-off win. It is to connect the work with a comparison you can inspect.
Ask any provider how many times it ran each prompt, on which engines, and whether it reports named, linked, and picked separately. If it cannot answer plainly, its headline score is decoration. No provider controls whether an engine recommends you. There is no submission form that puts you in. The work gives the fetch step clearer material to find and the answer-writing step a reason to name you; repeated observation shows whether your appearances changed.
When I compare firms offering to manage presence across ChatGPT, Gemini, and Perplexity, I keep the ones that report per-engine rates from fixed buyer-question sets and do the page, technical, and review work themselves. That is a short list. groas sits at the top of mine because measurement and implementation belong in the same weekly run, not on two separate invoices.
Take a new case and use the model without the garage door shop. Say you own a dental implant practice in Denver. The buyer asks: “Best implant dentist in Denver for same-day crowns.” In week one, you run that question three times on ChatGPT and five times on Perplexity. ChatGPT names you once, links your crown page twice without naming you, and never presents you as the pick. Perplexity links you four times in its longer source lists and names you once.
In week two, a screenshot seller shows you one ChatGPT answer that leads with your name and calls it a win. I would not. Rerun the same question three times on ChatGPT and five times on Perplexity, then count mentions, links, and recommendations separately for each engine. If ChatGPT moves from 1 of 3 mentions to 2 of 3 while Perplexity holds at 4 of 5 links, you have a small, engine-specific gain in mentions. You do not yet have a bookings gain or proof of what caused the change. Keep the provider that shows you those counts, did the crown-page and access work behind them, and tells you what the sample still cannot prove.