Your prospective Meta manager has a 4.2x ROAS screenshot. Meta calculated the number, sold the impressions, and decided which sales to claim. The number worth hiring on is incremental revenue against a holdout. A good manager will offer to test it in month one.

I used to put those screenshots in decks. Say a client spent $20k in a month and Ads Manager showed $84k in attributed sales. I would write 4.2x. The division was correct; the conclusion was doing more work than the math allowed. Some buyers might have clicked and purchased anyway. Others might have seen an ad, then bought after an email or a search. Returning customers might have checked out after seeing retargeting. Meta recorded the sale. That alone did not tell me whether Meta caused it.

This is a lab notebook, not a buyer’s guide. The question: How many dollars of revenue would disappear if those buyers did not see Meta ads? Below is a setup you can ask a manager to run on a Shopify store in the first 60 days. It has a control, a measurement plan, and a decision at the end. If a manager will not agree to one, you know what their ROAS screenshot can—and cannot—prove.

First, identify what Meta’s ROAS can’t tell you

Meta’s default attribution is 7-day click plus 1-day view. Someone can scroll past a video without clicking, buy later that day after a Klaviyo email, and still appear as a Meta-attributed sale. That is not fraud. It is credit. But make budget decisions from that column alone and you can count revenue another channel helped deliver as if Meta delivered it by itself. I used to call the platform generous. I was wrong to stop there. It was grading its own homework.

Retargeting makes the distinction harder to ignore. Geo-test summaries put the gap between platform-reported ROAS and measured incremental ROAS at 1.5x to 3x, widest on brand search and retargeting; they put retargeting’s incremental ROAS 40–70% below reported ROAS. The mechanism is plain: show an ad to someone with a cart open, then claim the checkout. In one practitioner holdout, paused-market revenue fell by only about one-third of the revenue Meta claimed it drove. That suggests much of the claimed revenue would have arrived anyway.

If most of your reported 4x comes from retargeting, ask whether you hired a salesperson or a cashier. The holdout is how you find out.

Define the numbers before anyone touches the account

A manager can improve an Ads Manager column without improving the store. To keep those outcomes separate, agree on three views of performance before the test starts:

  • Platform ROAS: Meta-attributed revenue divided by Meta spend. Use it to examine what the platform reports for a campaign or ad set, not to declare that every attributed sale was caused by an ad.
  • Blended ROAS: Total revenue divided by total ad spend across channels. Use it to see whether the wider advertising program is moving with the spend.
  • New-customer ROAS: First-order revenue divided by the spend you are evaluating. Use Shopify’s new-versus-returning split so repeat purchases do not quietly become an acquisition success story.

Each answers a different question. None, on its own, replaces a holdout. Imagine a brand raises Meta spend from $15k to $40k while in-platform ROAS holds at 3.8x and blended ROAS barely moves. That is a reason to investigate whether Meta is taking credit for demand also reached through Google or email, not a reason to celebrate the 3.8x. In an audit of 471 Shopify stores, median blended ROAS was 2.1x and median platform-reported ROAS was 3.4x. Those medians are not a conversion formula for your store. They show why the two columns should not be treated as interchangeable.

The new-customer split catches another problem. In the example math agencies pass around, $100k of total revenue on $25k of spend looks like 4.0x. If $60k came from new customers, new-customer ROAS is 2.4x. An operator teardown found an account showing 4.2x in-platform but about 1.6x on genuinely new customers after checking backend orders and removing retargeting buyers judged likely to have purchased anyway. The split does not, by itself, prove what Meta caused. It does stop a pitch from hiding acquisition behind repeat orders.

Have the manager show all three numbers, then test causation separately. If margin rather than revenue is the store’s binding constraint, this breakdown of an ecommerce brand’s shift to margin-aware bidding is the relevant companion read. For this protocol, keep the question narrower: what revenue changes when Meta is on versus off?

Protocol: make the sale survive a control

Question: Did Meta cause additional revenue, or merely record sales that would have happened anyway?

My expectation: Reported ROAS will overstate incremental ROAS by half or more when view-through credit and retargeting pick up demand that email, search, or earlier purchases already created. That is a hypothesis, not the result of your test. I want a control group that does not see Meta ads and a test group that does. If the revenue difference does not support the expectation, change the expectation.

1. Choose the holdout you can actually run

You have two routes. If the account has access to Experiments and enough spend, ask the manager to run a Meta Conversion Lift study. Meta recommends a minimum budget of $5,000 and at least 28 days to reach significance. The comparison is between users exposed to the test and users held out.

If that route is not available, set up a DIY geo holdout. Choose three to five reasonably matched markets you can leave without Meta for three weeks. Keep Meta running in the test markets and turn it off in the holdout markets. Then compare Shopify revenue in each group against its own baseline, adjusting for their starting sizes. A geo-test playbook describes the same on-versus-off logic with California against the rest of the country, or BC as a test market in an otherwise dark Canada account. Those are examples of the design, not markets you must choose.

Pick the markets before seeing the result. Write down why they are comparable, which campaigns will be off, and who can change that setting. Do not rescue a disappointing week by swapping the control geos. That changes the question halfway through the experiment.

Practical rule: choose the control you can keep dark and document, not the one that makes the most impressive slide.

2. Freeze the other moving parts

A holdout is not useful if you change several things at once. Before launch, put these controls in the manager’s working brief:

  • Keep the promo calendar and email flows unchanged during the comparison.
  • Do not shift the held-out Meta budget to Google or another channel.
  • Keep the agreed Meta spend in the test markets steady enough to read the comparison.
  • Record any change you cannot avoid in the daily log, rather than pretending it did not happen.
  • Export Shopify orders with first-order flags and UTM history on day zero, before anyone argues over which customers count as new.

I know a store cannot put the whole business in a freezer. The point is to avoid an obvious alternative explanation for the revenue difference. If a promotion launches only in the Meta-on markets, you have tested a promotion and Meta together. If held-out spend moves to Google, you have tested a channel swap. Neither answers the hiring question.

The holdout must be dark, not quietly funded through a different dashboard.

3. Run the window and log the inputs

Plan on 28 days minimum for the Meta lift route. For a geo holdout, use 21 days if you cannot afford a full month dark; practitioners have used three weeks in matched markets to see a pattern. That gives you full weekly cycles while keeping the control in place long enough to examine the result. It does not guarantee a decisive answer if sales are thin or the markets move differently for reasons unrelated to Meta.

Say you normally spend $20k a month across the markets in question. You might keep $15k planned for the Meta-on markets and leave markets that would have received the other $5k dark. Do not describe the unspent $5k as test spend, and do not reallocate it. The comparison needs a real absence of Meta in the control.

Log the same inputs each day, by market where the geo test requires it:

  1. Shopify revenue, including the pre-test baseline and revenue during the test.
  2. Ad spend by channel, with Meta spend in the test markets recorded separately.
  3. New-customer revenue from Shopify first-order data.
  4. Any unavoidable change to promotions, email, site experience, or budgets.

Revenue should come from Shopify rather than Meta’s attribution column. Spend comes from the ad accounts. Keep the source of each number beside it in the sheet. That sounds painfully ordinary because it is. A control with a messy ledger is just a more expensive argument about screenshots.

If a manager says three weeks is too long to leave money on the table, ask what 11 more months of unmeasured spend might cost.

4. Calculate the lift before opening Ads Manager

For matched markets with the same pre-test revenue, the simple version is:

Incremental ROAS = (revenue in Meta-on markets − revenue in Meta-off markets) ÷ Meta spend in the on markets.

If the markets started at different sizes, adjust the comparison for their baselines before taking the difference. Write down that adjustment before the run. Otherwise a larger market can look like lift simply because it was larger all along.

Here is illustrative math, not a reported test. Two matched sets of markets each did $50k before the holdout. During it, the Meta-on markets do $58k and the Meta-off markets do $52k. The difference is $6k. If Meta spend in the on markets was $4k, incremental ROAS is $6k ÷ $4k = 1.5x, even if Ads Manager reports 3.4x for that period. The platform number says which sales Meta claimed. The holdout comparison estimates what changed when Meta was present.

That distinction matches the direction of the paused-market finding and the operator teardown, but neither tells you your store’s result in advance. My expectation for a first run is incremental ROAS at 40–60% of reported ROAS. Put that expectation in the notebook before the results arrive. Do not retrofit it to whatever number the manager wants to present.

If the result is higher, investigate what is working and consider scaling it. If retargeting-heavy markets show little incremental lift, cut retargeting first and test whether prospecting or creative work earns the spend. If the comparison is too noisy to support either move, say so and plan the next control. A number you cannot read clearly is not permission to call platform credit incremental revenue.

Put the protocol in the first 60 days of the hire

Do not hire on promised ROAS. Hire on willingness to be measured. I would put the commitment in the scope of work, in plain English: run a holdout in month one; report blended ROAS from Shopify, new-customer ROAS from first-order data, and incremental revenue from the control; keep the agreed spend and terms month to month until the result can inform the decision.

The working calendar is short enough to discuss on the first call:

  1. Days 0–7: Lock the markets or lift-study setup, export the order data, agree on the baseline, and freeze the controllable changes.
  2. Days 8–35: Run the 28-day comparison. Keep Meta on in the test and dark in the holdout; do not move the held-out budget to Google.
  3. Days 36–45: Calculate incremental revenue per dollar, examine new-customer revenue, and decide what to keep, cut, or test next.

A manager can need time to learn an account. Fine. The holdout is part of learning it, not something to postpone until the reporting deck looks nicer. This is also why I would run Meta through groas: its flat $1,499 monthly Meta Ads fee covers work on audiences, budgets, and creative testing around the clock. The fee does not rise with spend, so the pricing model does not reward a larger attributed-revenue screenshot with a larger percentage cut.

Read the report for evasions, not just numbers

Ask for a sample report before signing. Ads Manager screenshots showing 7-day click and 1-day view, with no Shopify total or new-customer split, show you credit rather than cause. If retargeting takes 60% of spend and claims 80% of revenue, ask how the manager will test whether those buyers needed another ad. Do not let a polished chart answer a question it never measured.

Then ask the operational questions. Which geos will be held out? Who will verify they stay dark? What will happen if an email flow or promotion changes? Where will first-order revenue come from? If the manager cannot name the control, refuses to freeze avoidable changes, or insists a $30k study is the only possible route without discussing a practical geo holdout, the claimed ROAS is being protected from a test.

The manager worth hiring offers the control before you ask for another screenshot. That is the standard, not a promise that every first run will produce a tidy answer.

The result changes the hire either way. If incremental ROAS clears your blended target and new-customer ROAS holds above 2x, you have a reason to scale prospecting, keep retargeting capped, and pay for the work without arguing over attribution columns. If incremental ROAS lands near 1x while Meta reports 3x or 4x, start by cutting retargeting and testing creative or Google capture instead. Do not renew a percentage-of-spend deal on the strength of a credit number the control did not support.

I want the holdout because I am tired of arguments about columns. It may support the manager’s pitch, contradict it, or show that you need a cleaner run. All three outcomes beat hiring on a screenshot Meta generated for itself.

One last operational detail: export Shopify orders with first-order flags and UTM history on day zero, then keep that file with the daily spend log. When day 36 arrives, nobody should have to negotiate what new customer meant. The hiring decision can be hard. Finding the inputs should not be.