Incrementality Testing on Meta and Google: A Step-by-Step Geo-Holdout Playbook

Key takeaways

Geo-holdout tests are the cheapest causal-lift measurement most D2C and B2B brands can run. Pick matched regions, pause spend in one for 2 to 4 weeks, measure the difference. Measured’s playbook is the most-cited public methodology. Northbeam sees similar test design across portfolios. Tests cost 5 to 20% of channel revenue for the test window. Worth it 3 to 4 times per year on the largest two channels.

Most teams talk about incrementality. Few run real tests. Geo-holdouts are the lowest-cost causal-lift method available. A well-designed test takes 4 weeks and gives you a real answer about whether a channel is creating revenue or harvesting it.

The principle is old (econometrics 101) and the implementation is now accessible. Below is the playbook we use with D2C and B2B clients before they spend on more complex measurement infrastructure.

What is a geo-holdout test?

A geo-holdout test pauses paid spend in matched geographies for a fixed window, then compares revenue in the paused regions to the control regions. The lift difference estimates incrementality.

Different from platform-reported ROAS. Platform reporting answers ‘who saw the ad and bought?’. Geo-holdout answers ‘would those people have bought anyway?’. The second number is the one that matters for budget decisions.

Two design variations are common. Matched-region holdout pauses entirely in one set of regions. Spend-uplift test increases spend by 30 to 50% in test regions to estimate diminishing returns. Both work; the holdout version is easier to interpret cleanly.

How do you set up the test?

Setup needs three things: matched regions, a test window of 2 to 4 weeks, and pre-registered metrics. Pick regions that look alike on baseline revenue and recent growth, not by name.

  1. Pick matched region pairs. Similar baseline revenue, similar growth, similar seasonality.
  2. Choose the test window. 2 weeks minimum, 4 weeks better.
  3. Pre-register the metrics. Conversions, revenue, brand search.
  4. Hold all other channels constant. Otherwise you are testing the bundle.
  5. Run the pause cleanly. No leaked spend, no exceptions.
  6. Document the test plan before the pause window starts.

How big does the test need to be?

Sample size depends on baseline conversion volume and the lift you want to detect. For most D2C brands, holding out 15 to 20% of revenue in regions for 3 weeks detects a 10 to 15% lift with 80% power.

  • Below 30 conversions per week in the holdout. Underpowered.
  • Hold-out share 10 to 20% of total channel revenue.
  • Test window 14 to 28 days.
  • Significance threshold 80 to 90% power, 95% confidence.
  • Run on the largest two channels first; smallest later or not at all.
  • Pre-register sample size at planning, not after results land.

How do you read the result honestly?

Read the result as a range, not a point estimate. Honest output: ‘The channel was responsible for X% to Y% of revenue with 90% confidence.’ Anything tighter is fake precision.

Platform reporting often shows ROAS of 4x. Geo-holdout often returns true incremental ROAS of 1.6 to 2.5x for mid-funnel paid social. That gap is the platform’s pat-on-the-back versus reality. Both numbers matter; treat the platform one as a leading indicator and the geo-holdout one as the true lift.

The honest test will sometimes return a result the team does not want to hear. That is the test working. Resist the urge to re-run until you get a more flattering answer. The point of incrementality testing is to learn the truth, not confirm the budget.

How often should you run geo-holdouts?

Run on each of the largest two channels every quarter. Run on a third channel annually. Avoid quarterly tests on small channels; the math will not support the read.

  • Largest two channels. Quarterly.
  • Third largest. Annually.
  • Smaller channels. Skip. Use platform reporting plus self-reported attribution.
  • After major creative or audience changes. Re-test sooner if structure shifted.
  • Maintain a results log. Year-over-year trends are the highest signal of all.
  • Communicate to the team that holdouts are normal, not punishment.

How does the geo-holdout result feed the budget cycle?

Feed geo-holdout results into the quarterly budget review with two numbers per tested channel: incremental ROAS range and confidence level. Move budget when the range deviates meaningfully from platform-reported numbers.

  1. Compile last quarter’s geo-holdout findings.
  2. Compare incremental ROAS range to platform-reported ROAS.
  3. Identify channels where the gap is largest (positive or negative).
  4. Reallocate budget toward positive-gap channels.
  5. Set the next quarter’s geo-holdout test plan.
  6. Document the decision in the quarterly review minutes.

What pitfalls kill a geo-holdout?

Five common pitfalls: badly matched regions, contamination from cross-region behaviour, mid-test changes, weak statistical power, and confirmation bias in the read.

  • Badly matched regions. Match on baseline, not name.
  • Contamination. Customers in one region buy from another; track carefully.
  • Mid-test changes. Discipline matters.
  • Weak power. Hold out enough to detect the lift you care about.
  • Confirmation bias. Pre-register the interpretation criteria.
  • Seasonal contamination. Test windows during BFCM or sale events are noisy.

What does a 90-day incrementality rollout look like?

Month 1: design the test on the largest channel. Pre-register regions, metrics, and significance threshold. Brief the team on the change-freeze.

Month 2: run the test. Resist mid-test changes. Document any deviations as part of the run.

Month 3: read the result honestly. Present the range, not a point estimate. Reallocate budget based on the gap between platform-reported and incremental ROAS.

What are the most common implementation pitfalls?

Three pitfalls: badly matched regions, contamination from mid-test changes, and confirmation-bias reads when the result is ambiguous.

The cure is pre-registration and discipline. The test only works if the protocol survives the inconvenient findings.

What does the test look like in a real D2C account?

A premium D2C brand at 9M EUR revenue ran a geo-holdout on Meta in Q2 2026. Matched two pairs of NL and BE regions on baseline revenue and recent growth. Held out 18% of Meta spend in test regions for 21 days. Pre-registered metrics: revenue, conversions, branded search. Meta-reported ROAS in the test period: 3.8x. Geo-holdout incremental ROAS: 2.3x with a 90% confidence interval of 1.9x to 2.7x. The team reallocated 14% of Meta spend toward brand work in the next quarter. Quarterly geo-holdouts now run on Meta and Google. Year-over-year, the gap between platform-reported and incremental ROAS has stabilised at around 35%, which the team treats as the working calibration.

Quick reference checklist

  • Match regions on baseline revenue and growth, not name.
  • Hold out 10 to 20% of channel revenue.
  • Test window 14 to 28 days.
  • Pre-register metrics and significance threshold.
  • Need 30+ conversions per week in the holdout.
  • Report as a range with 90% confidence.
  • Run quarterly on the largest two channels.
  • Annually on the third channel.
  • Maintain a results log for year-over-year trends.
  • Resist confirmation-bias reads when ambiguous.

Frequently asked questions

Can I run a holdout on Meta and Google at the same time?

Yes but in different regions. Otherwise the bundle effect contaminates the read.

Will Meta let me hold out specific regions?

Yes via campaign-level geo controls.

What about Pinterest or TikTok holdouts?

Possible. Need enough volume per region; usually only at scale.

Should I run the test myself or hire a vendor?

Either. Discipline matters more than the operator.

How does this differ from MMM?

Geo-holdouts are causal, channel-specific, and slow. MMM is correlational, system-wide, and slower.

How often is too often?

Quarterly is the realistic cadence. Monthly burns budget and team patience.

How does this fit with the wider attribution stack?

It is one of the three layers. See post-last-click triangulation.

Want help applying this in your account? Request an incrementality pilot quote.

Stay Ahead of the Growth Curve

New research on AI search, B2B growth and conversion, straight to your inbox. No spam, unsubscribe anytime.