Most agencies will show you a ChatGPT ROAS number in the first call. I ask for the order IDs behind it, then ask a harder question: which of those purchases would not have happened without the ads? Reporting a purchase and causing one are different jobs. Only the second grows a store.

Myth 1: “An attributed purchase is an incremental purchase”

People believe this because the logic looks airtight on a screen. A shopper clicks a ChatGPT ad, the pixel fires, and an order appears in the dashboard with revenue attached. I used to present screenshots like that to clients as proof. I was wrong.

Attribution records that a sale happened after an ad. Incrementality asks whether it happened because of the ad. One counts presence; the other tests cause against a comparable group that did not see the ads. Platforms can over-attribute by an estimated 20–60% once that baseline is subtracted. That is how a tidy 3x attributed ROAS can become something like 1.5x incremental ROAS.

Free tool: the ROAS Calculator works out your return on ad spend, your break-even ROAS and the budget a revenue goal takes.

Some credited orders would have come through Google, email or direct anyway. Ask the agency what comparison could show otherwise: a geo holdout, a paused audience, or a clean new-customer read. Ask when that comparison will happen, too. If the answer is always “after we scale,” the ROAS number is making the budget decision before anyone has tested what it means. None of these checks is as convenient as a dashboard screenshot. That is rather the point. If the dashboard is the test, keep your budget where it is.

Myth 2: “Google, Bing and ChatGPT revenue can simply be added together”

This one survives because each platform sends its own tidy report. Google claims the order. Bing claims the order. ChatGPT claims the order. Add the revenue columns and the total looks like growth.

I ran accounts that way for years before a Shopify reconciliation embarrassed me. The mechanism is simple: a shopper who touches more than one channel before buying can appear in more than one report for the same purchase. One study of 200-plus ecommerce accounts found 1.6 claimed conversions per actual sale when self-reported numbers were summed. In its worked example, cross-platform double-counting alone took a reported 4.0x ROAS to around 2.59x.

ChatGPT adds two wrinkles worth putting in the measurement spec:

Reconcile platform claims against unique store orders first. Keep the platform claims visible, but do not mistake their sum for store revenue. Then argue about performance. I wrote about the same counting gap in Meta Says 4x ROAS, Shopify Says 1.8x: Which Number Changed? The line for your agency call is shorter: show me one order table, not three ROAS screenshots.

Myth 3: “A returning customer’s order proves new-customer acquisition”

Returning buyers make an acquisition report look good. They often convert at higher rates and order values, so a ChatGPT report heavy on existing customers can print a handsome ROAS while adding little new demand. The example I keep taped to my monitor: 90% returning at 6x ROAS is often worse for acquisition than 40% returning at 3x. The first number may describe harvesting; the second may describe growth.

If the agency cannot split first orders from repeat orders, its acquisition ROAS is a loyalty report wearing an acquisition costume.

What counts instead? Separate new-customer revenue, a first-order count and a new-customer acquisition cost target. Add exclusions and a periodic retargeting holdout to check which customers would have bought anyway. Do not assume ChatGPT can match identities across platforms for you. Its ads are matched to conversational context rather than search keywords, with a sponsored unit under the answer for Free and Go users. That describes placement, not whether the buyer is new. An order can be real, profitable and still fail the acquisition test. Ask for first-order count by order ID before accepting an acquisition ROAS.

Myth 4: “Revenue ROAS tells you whether the channel is profitable”

I lost money on a 3.2x ROAS account once, and the client thought I was joking. Revenue had cleared, the card was charged, and the dashboard glowed green. Then the store did the math the dashboard did not: cost of goods, outbound and return shipping, payment fees, a 15% promo code, and a return rate that ate one order in eight.

Revenue ROAS is revenue divided by ad spend. It does not subtract what it costs to fulfil and keep the order. For a contribution-margin calculation, break-even ROAS equals 1 divided by contribution margin. At a 25% margin, break-even is 4.0x. The same reported 3.0x loses money at 15% or 30% margin and turns a profit near 50%, before any costs the margin figure leaves out.

The gap can widen when the purchase value sent to a platform is a pre-discount cart total, refunds arrive later, or the management fee sits outside the report. The pixel can only report the value it receives; it cannot tell you what the store keeps. Judge the channel on retained revenue after discounts and returns, minus variable costs and the fee. Ask for one order reconciled from reported revenue to margin before scaling on ROAS alone.

Ink cartoon of a store owner celebrating a 4x ROAS report while profit leaks out the back door

Myth 5: “A small-budget test can settle the ROAS question”

This myth is tempting because ChatGPT clicks are expensive and nobody wants to gamble. Say you spend $500 at a $4.41 CPC, a level one high-AOV store paid across seven months of ChatGPT Ads tests. That buys about 113 clicks. At a 2% conversion rate, you get roughly two orders, maybe three. One extra order can swing the ROAS dramatically. One refund can erase it. I have watched clients argue for an hour over a change that came down to one customer changing their mind.

Volume matters more than budget bravery. As a comparison, Meta testing guidance calls for around 50 conversions per week per ad set, with 25 to 50 needed to spot trends. That is not a ChatGPT rule; it illustrates why two purchases are a thin basis for an economics decision. ChatGPT results bring their own noise: the same campaign paid $5.10 per click in the UK versus $22.89 in New Zealand, and panel clicks outnumbered GA4 sessions 57 to fewer than 20 in one case. Even the denominator deserves a look.

A small test can show whether access, delivery and tracking work: whether the feed is live through SFTP, the pixel and Conversions API fire, and purchase values pass in the intended units. It cannot settle whether the channel earns its budget on the strength of two orders. Use add-to-cart or checkout-start as provisional signals while purchases accumulate, and set the purchase threshold for an ROAS decision before the test begins. Otherwise, every additional order invites a new interpretation of the same tiny sample. A working test is not yet a profitable channel.

Myth 6: “One agency means one comparable measurement system”

This is the myth I find hardest to kill because it sounds like professionalism. One agency, one dashboard, one ROAS you can compare across Google, Bing and ChatGPT. I wanted that to be true when I managed all three by hand. It is not. An agency logo cannot make three reporting systems count the same way.

ChatGPT Ads is matched to conversational context, not keyword searches. Its conversion objective counts results against a target CPA; there is no published ROAS-target bidding, and the default Maximize results setting offers no cost guarantee. Launch also depends on an OpenAI product feed through SFTP, plus pixel and Conversions API tracking. Different eligibility, bidding and reporting windows make a blended number especially easy to sell and especially hard to interpret. Even when case studies appear without spend, dates or sample sizes, the missing detail does not become evidence because it sits in an agency deck.

The fix is not a prettier dashboard. It is one written measurement spec across every channel: the same purchase definition, the same store-order reconciliation and a decision rule agreed before spend starts. I ask an agency to put five things on one page:

  1. Budget cap by channel. If you spend $20k a month across search, how much can go to ChatGPT, and who approves a shift? Without a cap, money can follow the best-looking report, which may also be the most over-attributed one.
  2. Purchase definition. Which store order ID counts, and when are cancellations, discounts and refunds removed? A panel click, a session and a pre-discount cart value are not interchangeable with a retained order.
  3. Reconciliation method. How will Shopify or WooCommerce orders be matched against Google, Bing and ChatGPT claims by order ID, click ID and date? A ChatGPT claim that cannot be tied to a unique store order does not belong in reconciled ROAS.
  4. Decision window. Which click and view windows apply, how are conversions dated, and how long can a purchase lag before it counts? A 30-day click plus 1-day view report and a 7-day click report make different claims.
  5. Stop conditions. Which CPA or new-customer CPA pauses ChatGPT, and how many reconciled orders are needed before scaling? No threshold means every disappointing report can become a request for more time.

The page matters because someone has to use it when the numbers look good, not just when they look suspicious. This is why I prefer one operator working across the channels under the same spec to three reports fighting for the same order. groas runs Google, Bing and ChatGPT Ads as one Search Ads service for $1,499 a month, regardless of spend, with bids, keywords and budgets worked around the clock. A flat fee does not rise when an attractive ROAS report persuades you to raise spend; a percentage-of-spend fee does. That distinction matters when you ask an agency to challenge its own numbers. groas also shows the reasoning behind its actions in plain language. I would still put the purchase definition and stop rule in writing. The action log is where I would check whether the work follows them.

The next time a deck shows a blended 5x, remember the hardest myth: one logo did not make Google, Bing and ChatGPT count the same way. Ask which orders in your store would not have happened without ChatGPT, how the agency will test that claim, and how it will reconcile every credited purchase by order ID. Get the rules in writing before you approve the spend. Then let the budget follow the reconciled table, not the logo on the slide.