LinkedIn
A page gets cited in an AI Overview a week after you rewrite it. Most advice calls that a win. I call it an unanswered question: would Google have cited the page anyway?
I used to make the same mistake with ad copy. Run a change without a control, watch sales move, take credit. Run it properly with a holdout, and sometimes the win disappears. AI Overviews make that habit especially costly: 40–60% of cited domains change every 30 days even when you touch nothing. If you cannot separate your edit from Google’s reshuffle, you have not learned anything. Here is the 21-day test I would run.
The question: did your edit beat the reshuffle?
Write the question on the first line of the sheet: Did this specific page edit increase citations relative to similar pages I left alone? Not “Did citations rise?” That is a different question, and an easier one for random movement to answer.
Overviews drift by more than 59%, and 70–90% of citations turn over across six months. The answer can change as the system draws on different chunks at query time. Practitioners make the same point about classic rankings: a before-and-after comparison on the same pages cannot isolate your change from seasonality or algorithm updates.
My expectation is narrow. If an answer-first edit gives Google a clearer passage to cite, edited pages should gain citations more consistently than comparable untouched pages. Both groups may rise or fall; the difference between them is what matters. A citation after an edit is an observation, not a verdict.
Setup: pair the pages, then freeze the queries
Pick 6 to 10 landing pages that already resemble one another. Match by template, buyer intent, and similar authority. Pair two service pages for nearby cities, two product comparisons, or two use-case pages. Within each pair, flip a coin: one page becomes the test page; its match stays untouched as the control. Changing one arm while leaving the other alone is the split-test structure you need here.
Next, freeze 20 buyer queries from Search Console, sales calls, and autocomplete. These should be questions a buyer might use to find those pages, not 20 variations of your brand name. Before the test starts, note which page pair each query is meant to probe. Otherwise, a citation for an unrelated page can make your site-level number look better without saying anything about the edit.
Run the same fixed sample on the same cadence, and record whether an Overview appeared separately from whether it cited one of your pages. Date the query list. Do not add a promising new query halfway through because it makes the graph prettier. The queries and page assignments are part of the control.
Choose one edit you can name in a sentence
If you change four things at once and citations move, you cannot tell which change mattered. Choose one edit and apply it to test pages only. Candidates include:
- An answer-first paragraph of 40 to 60 words that states the conclusion plainly.
- One question per H2, with a self-contained answer beneath it.
- A dated fact block with named sources inline.
- Consistent business, city, and product names on local or B2B pages.
I would start with the answer-first paragraph. The mechanism is legible: if the Overview needs a passage it can use, the page now offers one without making the system piece an answer together from scattered sentences. Entity clarity would be my second choice for local and B2B pages. Test it separately; do not bundle it with the paragraph edit and call the package a discovery.
I would not start with schema. Google sets no schema requirement for citations, and Search Atlas found similar visibility across 0 to 100% schema coverage. Test the words the buyer reads before the code they never see. One variable means one variable.
Days 1–7: build a baseline before editing
For seven days, check the 20 queries at the same time of day from the same setup: logged-out browser and fixed location. Record what appeared, not what you hoped to see. Overview presence and a citation of your domain are separate readings.
For every check, log:
- The query and date checked.
- Whether an Overview appeared.
- All cited domains, in the order shown.
- Whether the cited URL was a test page, its control, or neither.
- A screenshot URL or saved HTML for that reading.
Look at each pair’s pre-trend before you touch a page. If one page is regularly cited and its match never is, the pair may tell you little about a later gap. Replace it with a closer match or drop the pair before day 8. Note the decision on the sheet. Do not quietly discard an awkward control after seeing the result.
A single cited-or-absent reading is noise. This week tells you whether the two arms were behaving similarly before you intervened. It also makes the work less glamorous than the case study version. Good. No baseline, no credible before-and-after claim.
Day 8: edit the test pages and leave the controls alone
Apply the chosen edit to the test half. Do not fix a typo on a control page while you are there. Do not let a sitewide header change slip into the test unnoticed. If something else changes, log it. Your job is to preserve the comparison, not to keep the account looking tidy for three weeks.
Request indexing in Search Console for each edited URL and record the request date. A request does not guarantee indexing; requesting the same URL again does not jump the queue. Check whether Google has recrawled the edited pages before treating a flat result as evidence against the edit. If Google has not seen the new version, the experiment has not reached the system you meant to test.
During weeks 2 and 3, repeat the baseline check. Same 20 queries, same time of day, same logged-out setup, same page assignments. Same sheet, same hour, zero improvisation. That rigidity is the method, not paperwork attached to it.
Split-test guidance recommends at least 2 to 4 weeks and a check that test and control trended together beforehand. This protocol gives you one baseline week and two weeks after the edit. It is a practical, fixed read, not a promise that 21 days will settle a small effect. If a core update lands during the test, mark the dates and keep logging. It may affect the arms differently, so do not wave it away by saying the control handled it. Read the gap with that disruption in view.
Measure citations first; treat clicks as a separate question
Count citation rate for each arm. On each day, what share of the 20 frozen queries cited a test page? What share cited a control page? Keep the query-to-page assignments beside those totals so you can inspect whether a gain came from the pages and questions the edit was meant to affect.
Suppose test pages go from 2 of 20 queries to 6 of 20 while controls stay at 2 of 20. That is worth investigating. If both arms go from 2 to 6, the movement is not evidence for your paragraph. The numbers are an illustration, not a result from a run. On a small site, the same logic works as a change-five-pages, freeze-five holdout.
Record the actual cited URL, not just a yes beside your domain. Was it the edited page, its untouched match, a competitor, a forum thread, or a YouTube transcript? If the citation lands on your control page, do not award the point to the edit. If forums keep taking the queries, that may be a source-type problem rather than something an answer-first paragraph can fix. A domain-level lift can hide a page-level miss.
Read clicks second, through a proxy. Search Console does not separate AI Overview impressions or clicks into their own source, and GA4 supplies no AI Overview referral parameter, so you cannot pull a clean Overview click report. Watch Search Console impressions and CTR for test URLs against control URLs instead. Do not turn that proxy into an exact citation-to-click attribution.
And do not turn a citation into revenue in your head. In one cited analysis, position one lost 58% of its CTR when an Overview appeared, although cited brands still earned more clicks than uncited brands in the same data. Citation rate tells you whether you were named. The Search Console comparison helps you judge whether the visibility coincided with useful traffic. Those are two readings, not one.
Day 21: decide what the result permits you to say
Write the read date on the sheet before the edit. On day 21, inspect the final seven days rather than selecting the best afternoon. With 6 to 10 pages and 20 queries, I would not claim textbook statistical significance. Small tests struggle to detect small lifts; practitioners prefer dozens of pages with solid traffic, balanced pre-trends, and at least 2 to 4 weeks. Matching pages and fixing the read date make this test more disciplined. They do not make it enormous.
I would call the result a useful win only if all three conditions hold:
- Test pages gain citations relative to controls, rather than both arms moving together.
- The gap persists across the final seven days instead of appearing once.
- The cited URLs are the pages you edited.
Then check the three ways the result can fool you. Indexing lag means Google may not have seen the edit. Query volatility means an Overview may have changed for reasons unrelated to your page. Seasonality can shift demand underneath both arms. Check crawl status, read the daily record, and compare the matched pages. A control reduces guesswork; it does not abolish it.
If the result is mixed, say so. A day-21 gap on one query is not a rollout plan. Neither is a flatline on pages that were never recrawled. The point of the fixed date is to stop negotiating with the graph after you see it.
Give a vendor the test half, not the whole site
Use the same protocol when a vendor says its software will get your landing pages cited in AI Overviews. Give it the test pages. Keep the controls, 20 queries, edit variable, and read date fixed. If the agreed edit is an answer-first paragraph, judge that edit. Do not let a late H2 restructure or fact block blur the answer.
A fixed-query test with a control group helps distinguish a tool that changes what gets cited from one that only reports what happened. The first filter I apply is simple: does it help earn citations, or just report metrics? If a vendor objects to a holdout, I would not replace the test with a dashboard tour.
I run this kind of holdout inside groas because doing it by hand every month is miserable. The models edit the test arm and log what changed and why; the named strategist reads test against control on the date we set. Same discipline, without the 1am spreadsheet. If a vendor cannot show the test-versus-control gap, keep your money.
The next move depends on what day 21 says
Pick matched pairs, freeze real buyer queries, change one thing on half the pages, and read the result against the half you left alone. If the test pages pull ahead and the cited URLs match the edits, you have a reason to ship that edit wider. If both arms move together, you have a reason not to credit the rewrite.
If day 21 is flat, do not extend the test just to hope the graph bends. Check whether the edited pages were recrawled. If they were, choose the next single variable and run it on a fresh pair. Keep the sheet; I keep mine for a year so the next test can start with what the last one ruled out.
A win changes what you ship. A flat result changes what you stop paying for. Either is more useful than calling Google’s weekly reshuffle your strategy.