A folder of new ads is not evidence of creative testing. Before I accept a testing deliverable, I want a chain I can follow: a genuinely different hypothesis, actual delivery, spend, downstream results, and a decision someone owns. Two dozen thumbnails do not establish that chain, especially when half show the same footage wearing different on-screen text. Meta’s own announcement distinguishes iteration, such as identical visuals with different text or CTAs, from diversification: distinctly different assets aimed at different personas or use cases. Both have a place. They are not interchangeable, and an agency should not invoice you for one while presenting it as the other. Keep this checklist open during finalist calls and monthly reviews. The question is not whether the agency made something. It is whether the work answered something, and whether you can inspect the evidence.
Before signing: ask for the work behind the deck
-
Ask who produces the creative, who approves its claims, and who makes the next decision once results land. Get those responsibilities clear before discussing production volume. The “creative team” can turn out to be a media buyer with a stock-imagery account, a mismatch flagged by Powercouch Media. That is a different service from a testing program. The acceptance check is simple: the agency can explain who turns a hypothesis into an asset, who checks what the asset promises, and who decides what happens after delivery. A department name is not an answer to all three.
-
Request a redacted sample test log and monthly report from a live account. A performance deck is a highlight reel; a log should include the losers and unresolved tests, too. Ask for the hit rate: the share of tested ads that reached the agency’s winning threshold over 30 and 90 days, as suggested by Opascope. Then ask what that threshold means and which ads entered the denominator. The number is useful only if the agency can show how it was calculated. Accept evidence of a repeatable method, not a percentage floating above a row of attractive thumbnails.
-
Ask what share of the budget stays in a standing test lane, and what that allocation can realistically support. Meta’s November 2024 guidance recommends at least 20 diversified ads in Advantage+ shopping campaigns and 20–30% of the overall budget toward new ads. Ask how the agency applies that guidance to your account rather than treating those numbers as the deliverable. A production target still needs delivery, a judging rule, and enough spend to answer the question. “We test when there is time” is not a plan for replacing a winner when it starts to fatigue.
Before production: separate a hypothesis from an editing task
-
Write the customer objection or buying motivation in one sentence before anyone shoots anything. “We are testing hooks” describes a task, not a hypothesis. “Renters leave in the first three seconds because the video reads like an ad, so a price claim up front should hold them” makes a prediction that can fail. Ask what result would count against that prediction, not just what success would look like. If every possible outcome supports the original idea, the agency has written a rationale for making an ad. It has not written a test. Get the question into the log before production makes everyone attached to the answer.
-
Mark what changes between assets: opening hook, footage, format, message, or tone. Use one row per asset and one column per variable so the distinction survives beyond the briefing call. Mereny’s testing framework provides a useful vocabulary: a concept is a genuinely different idea, such as a new angle, format, or messenger; a variation modifies something, such as a hook line, edit, or thumbnail. I use concepts to organize the search for winners and variations to extend promising work. The acceptance check is whether the labels describe meaningful differences, rather than whatever makes the production total look largest.
-
Label text swaps on identical footage as variations, not distinct concepts. Two videos carrying “Text A” and “Text B” may test a message difference; they do not, by themselves, demonstrate creative diversification. That distinction matters because similar-looking assets can be grouped together by Meta’s ranking systems, as discussed by Engage Era. Do not reject iteration. Reject inflated accounting. If the agency ships thirty text swaps, the log should show which questions those variations address and what their results establish. Thirty exported files are thirty exported files. They are not automatically thirty different ideas about why someone might buy.
Before judging: establish that the test received delivery
-
Inspect spend and impressions by concept, not just by campaign total. Export ad-level delivery and group it using the concept labels from item 5. If one concept takes 80% of the budget, inspect what the others actually received before accepting a comparison. AdEye’s framework suggests 3–5 genuinely different concepts at a time, with even budgets and a run of at least 5–7 days or roughly 50 conversions per variant. Treat that as a planning reference, not proof that a test succeeded or failed. The acceptance question is whether each concept received enough delivery to support the verdict attached to it. A campaign total cannot answer that.
-
Separate an underdelivered ad from a demonstrated loser. An ad that spends $40 of a $400 test allocation has not automatically failed. Ask for a numeric kill rule agreed in advance: the spend floor, the relationship to target CAC, and the observation window. A discussion in r/FacebookAds about an agency logging 1,000-plus changes in three weeks illustrates the problem with constant intervention: activity can destroy the conditions needed to judge performance. The agency should explain why it stops an ad, not merely point out that it stopped spending. “Underdelivered” and “lost” belong in different rows of the explanation.
-
Ask how the agency separates concept exploration from a controlled comparison. Meta’s A/B test guidance calls for testing one variable with otherwise identical ad sets, avoiding overlapping audiences, and running tests for at least 7 days and no more than 30 unless customers convert slowly. A comparison of genuinely different concepts can change several creative elements; it should not then claim to isolate the effect of one hook. Likewise, creative running alongside established winners does not automatically produce a clean causal comparison. Require the agency to name the kind of test it ran and limit the conclusion to what that setup can establish.
Before declaring a winner: connect the result to the business
-
Connect clicks and leads to qualified enquiries or purchases wherever tracking permits. A form fill is not a customer, and creative that wins at lead level can lose at pipeline level. Name the downstream signal used for each verdict: purchase value, qualified enquiry, or booked call. Then ask how that signal reaches Meta, whether through Conversions API or an offline conversions import from the CRM. Commonly done wrong: the sales data stays in a disconnected CRM while the agency judges creative entirely on platform-reported leads. If the connection is missing, record that limitation. Do not let a lead-volume result quietly become a claim about revenue.
-
Record conversion lag and measurement limitations beside the result, not in a footnote nobody reads. Meta’s A/B test guidance says to extend the test window when the typical customer takes longer than 7 days to convert. A day-three verdict is premature for a purchase that takes two weeks of consideration. Ask when the test stopped receiving spend, when its results were judged, and whether the agency allowed for the expected lag. Those are different questions. The acceptance check is whether the observation window matches the buying process, rather than ending whenever the reporting deck needs a winner.
-
Reject a business winner declared on CTR alone. CTR is a diagnostic, not a substitute for the outcome named in item 10. Meta’s diversification announcement reports image-generation features producing 11% higher CTR and 7.6% higher CVR: even in that example, the metrics do not move in lockstep. AdRiseLab also notes that creative comparisons can reflect both user response and Meta’s delivery efficiency. Ask what the winner actually won on, and what remains uncertain. If the hypothesis concerns attention, a click metric may help answer it. That still does not make the ad a proven purchase or pipeline winner.
Before accepting the report: demand a decision and its receipt
-
Require one verdict per test: keep, retire, retest, or inconclusive, with the reasoning attached. A leaderboard of top creatives does not tell you what the agency learned or what it intends to do. “Inconclusive” is a legitimate verdict when delivery is too thin to support a decision; it should not be disguised as a loss to keep the table tidy. Check the reasoning against the original hypothesis, the agreed judging rule, and the business signal. A verdict that cannot survive those checks is an opinion with a metric beside it. Accept a decision with a cause, including an honest decision that the evidence does not yet settle the question.
-
Reconcile the monthly summary with the underlying change log. Every summarized test should trace back to its hypothesis, delivery, spend, and verdict; every spend spike should have an explanation. For the broader reporting review, use the separate monthly acceptance checklist. The creative-testing check is narrower: pick three rows at random from the summary and ask to see their entries in the log. Do not let the agency substitute its three favourite examples. The summary should compress the evidence, not replace it. If the trail disappears as soon as you leave the presentation, the deliverable is not ready to accept.
Before payment: put acceptance and ownership in writing
-
Define the first reporting period’s acceptance criteria before you pay. Specify the required log fields, scheduled tests, and business metric for each verdict. At an illustrative $20k a month, the agreement might call for four genuine concepts in a standing test lane by week three, a verdict for every closed test, and a trace from each verdict to spend and the closest available revenue signal. Then use a written hiring verdict: pass means the hypothesis, delivery, outcome, and decision all connect; conditional pass means one or two gaps need proof in the first reporting period; reject means the agency cannot show a log, judges everything on CTR, or mislabels text swaps and refuses to correct it. Missing evidence is an unanswered question, not proof of fraud. It is still not evidence you can accept.
-
Name the owner of every unresolved test, next action, and creative-policy decision. This is the item most often skipped. “The team owns it” leaves the decision owner unnamed. Claim approval and policy compliance belong inside the testing deliverable, not in an onboarding document that disappears after signature; this first-hand account of repeated account restrictions shows why the responsibility matters. Put a name beside the decision before a rejected ad, a killed winner, or a policy restriction starts the search for whoever was supposedly responsible. The cost is not just wasted test spend or interrupted delivery. It is months of unverifiable claims because nobody must explain the reasoning or own what happens next. Most of this checklist asks the agency to demonstrate a method. The final two items ask it to commit to one. An agency that refuses both has told you what you are buying: a folder of new ads.

