Your agency’s monthly SEO report is not proof anything useful happened. If you pay for ongoing SEO and AI search management, I want you checking evidence you can open, repeat or trace, not admiring a score you have to trust. These seven checks keep implementation, visibility and business results separate. Mixing them is how a report turns “we published something” into “we grew your business” without showing the work in between.
1. Open the pages they say they published
I used to tell clients to start with rankings. I was wrong. Rankings move for reasons your agency did not cause. A PDF that says “4 pages published” proves nothing until those URLs load on your domain and read like finished work.
Start with an evidence trail, not rankings: live pages, sitemap changes, Search Console queries you did not appear for before, and a dated log of shipped changes. Those are different kinds of evidence. A live URL shows that a page exists. A new query shows observed visibility. Neither, on its own, proves the page brought in a buyer.
Open every URL they list. Check that each page answers one real buyer question instead of restating another page in new words. If two pages could swap titles and nobody would notice, you paid for duplication.
Ask for three things:
- The live URL.
- The publication date.
- One sentence explaining which buyer question the page answers.
First step: request that list in one email, then open the pages. Do not let a content-production total stand in for an inspection. Four published pages can be four useful assets or four more things your next agency has to clean up.
2. Ask for the before-and-after of one technical fix
An audit recommendation is not work. I have seen 40-page audits where every ticket sat in “recommended” status while the client kept paying a retainer. The document got longer. The site stayed the same.
A fix counts when it is live on the URL where the problem was found. Re-check the same evidence after deployment, whether that means schema present, a canonical corrected or a robots rule removed. A fix-verification loop closes the gap between advice delivered and changes actually deployed.
Pick one fix from the report and ask for:
- What the affected page showed before.
- What that same page shows now.
- The date the change went live.
A relevant Search Console screenshot, page-source record or crawl record can provide the comparison. The evidence should match the problem, not merely show that someone opened a tool.
First step: choose one supposedly completed fix and ask to see it live. If they send a closed ticket instead, you have evidence of a workflow status, not yet evidence of a deployed fix. And even a verified fix proves implementation, not a traffic or revenue gain. Keep that boundary intact.
3. Follow claimed mentions and citations back to their sources
A logo wall in a report is not a citation. Click every claimed third-party mention and confirm the page exists, names you correctly and links where the report says it links.
For AI answers, make another distinction: a brand name in the response is not the same thing as a clickable link. ChatGPT names brands about 3.2 times more often than it links to them. “Mentioned in 40 answers” can therefore describe something quite different from 40 clickable citations.
Different engines fetch different sources, and answers get rewritten. A report needs to tell you what was observed, where and when, rather than flattening every appearance into coverage.
For each claimed result, ask for:
- The source URL, where a page is being claimed.
- The engine and observation date, where an AI answer is being claimed.
- Whether the result was a text mention or a clickable link.
First step: trace the claimed mentions back to the underlying evidence. If a source page now returns a 404, flag it as unavailable rather than counting it as current coverage. If it never named you, cross it off. A historical appearance and a link a buyer can click today are not interchangeable.
What the deck calls “coverage,” I call inventory until I can inspect it.
4. Repeat a small sample of the tracked AI questions
Pick five prompts from the agency’s tracker and run them yourself in the same engine. Record the engine, date, exact wording and result: named you, linked you or neither.
This is a spot-check, not a verdict. AI answers shift from run to run. In SISTRIX’s analysis of AI visibility, 70% of brands that appeared showed up on fewer than 1 in 5 days, while only 6% appeared on 4 in 5 days across the surfaces studied. One manual check that misses you proves very little. Repeated misses deserve a closer look, not an immediate accusation.
The broader measurement question is whether the tracker covers enough different buyer questions. In the same analysis, testing 20 different prompts was more than three times as accurate as testing one prompt for 20 days. Repeating a narrow sample does not make it representative of everything your buyers ask.
Ask the agency to show:
- How many distinct prompts sit behind the score.
- Which buyer questions those prompts represent.
- Which engines were checked and on which dates.
- How mentions and links are counted separately.
First step: ask for the prompt list behind the visibility score, then rerun five of those prompts. A small list is not automatically useless, but it is a narrow view. I would rather see that limitation stated plainly than watch twelve prompts dressed up as the whole market.
5. Inspect Google performance at the page-and-query level
Traffic totals lie by omission. A PR hit, a seasonal spike or your own brand searches can lift sessions while buyer discovery falls. The total can be accurate and still answer the wrong question.
Open Search Console’s Search results performance report and inspect queries alongside pages. Separate searches for your brand from non-branded discovery queries; this branded-versus-non-branded breakdown is more useful than treating all clicks as one undifferentiated win.
Then look at the pages the agency says it improved. Are they appearing for the buyer questions those pages were meant to answer? Are clicks changing on those pages, or is the headline increase happening elsewhere?
If non-branded clicks to your money pages are flat while total clicks rise, the total does not demonstrate stronger buyer discovery. It also does not prove the agency did nothing. It tells you where the explanation needs to get more specific.
Pair that inspection with GA4 before accepting an AI traffic claim. Look at the session source and medium behind reported referrals, rather than assuming a dashboard label explains the whole journey. Keep an identifiable referral separate from an impression, and both separate from a business result.
First step: choose the pages the agency claims improved and inspect their queries, clicks and key events. Ask what changed on those pages and what happened afterward. A rise in AI impressions is not proof of visits; a rise in visits is not proof of enquiries. Screenshots of exposure cannot do all three jobs.
6. Ask a buyer to use the content to make a decision
A page can rank, get cited and still fail at its job. I learned this on an ecommerce account where a buying guide pulled steady traffic for six months and produced almost no checkouts. The copy answered the general question correctly. It never answered the buyer question: will this fit my case, what exactly am I getting, and what do I do next?
Visibility without a decision is just publishing costs with better charts.
Hand the page to someone who buys what you sell and give them two minutes. Ask them to explain:
- Who this is for, and who should skip it.
- What the offer includes and what it costs to start.
- What the next step is if they want it.
If they hesitate, you have something concrete to investigate. You do not need another abstract discussion about whether the content feels “authoritative.” You need to know which part of the buying decision the page leaves unresolved.
This check does not establish how much revenue the page caused. It tests whether the content helps a buyer understand the offer and act on it. That is a useful judgment in its own right.
First step: run this on your two highest-traffic SEO pages this week. Fix fit, offer clarity and the next step before paying for five more pages with the same problem.

7. Trace a sample of enquiries back to the evidence you actually have
Pull your last 20 enquiries and sort them into three groups. Not everything belongs in a revenue chart just because somebody can draw an arrow to it.
- Observed referrals: a source and medium you can inspect in GA4, such as a referral from chatgpt.com or perplexity.ai.
- Customer-reported discovery: what a buyer tells you in a “How did you hear about us?” field.
- Inferred influence: a possible journey you cannot verify, such as an AI answer shaping a shortlist before someone finds you through Google.
The first two give you evidence of a recorded referral or a buyer’s account of discovery. They are not identical, and neither automatically establishes closed-won revenue. The third is a hypothesis. Keep it labelled that way.
I have watched agencies move every unattributed lead into the third group and call it AI revenue. That is how a mention becomes a closed-won claim with no path between them.
First step: tag one month of enquiries this way before you renew. If almost everything sits in the inferred group, you do not have demonstrated attribution. You have a story to investigate. That does not mean the work had no influence; it means the available evidence cannot carry the claim being made.
This is also where I would ask for the rest of the evidence package. A managed SEO and AI search service should hand over the following without an argument:
- Live URLs with publication dates and the buyer question each page answers.
- Change records with before-and-after evidence for technical work.
- Source links for claimed third-party mentions, with mentions and citations distinguished.
- The visibility method: prompt count, engines, dates and re-check rules.
- Business results labelled as observed, customer-reported or inferred.
These are not five versions of the same proof. They answer different questions. The distinction also runs through the seven AI SEO myths agencies sell: a service claim is not a substitute for something the buyer can inspect.
That is the standard I hold groas to as well. groas runs paid ads, SEO and AI search around the clock, with a named strategist accountable for the work and actions logged with the reasoning behind them in plain language. That model should make inspection straightforward. If the log says a page shipped, you should see the URL. If it says a citation was earned, you should see the source link and the engine that returned it.
I do not want a special standard for our reports. I want three separate answers from any provider, including groas:
- Implementation: did the work ship, and can I inspect it?
- Visibility: did observed presence change across the measured queries, prompts and engines?
- Business results: which enquiries and outcomes can we trace, and where does the evidence stop?
A provider that merges those answers into one score is asking you to trust the maths instead of checking the work.
If I had one hour with a new account, I would start with this seventh check. I would pull the last 20 enquiries and tag them before opening the agency’s report. Knowing which sources are observable and which ones buyers actually named tells me what the SEO and AI work needs to explain.
Say you are spending $20k a month, 14 of those 20 enquiries have branded-search or direct traffic sources, 2 buyers name ChatGPT, and none show an AI referral in GA4. Those observations should stay separate, not be forced into one attribution total. They give the other checks a purpose: inspect the pages, test the buyer questions and examine the visibility method to understand the gaps.
That is where I would spend the first hour. Not arguing about whether a visibility score went up. Finding out what the business can actually trace, then asking the provider for links that explain the work behind it.

