SEO ROI & Measurement · 4 min read
Can You Measure Online Ad Impact? The Limits of Attribution
Short answer
Partly. Attribution models assign credit for conversions that were tracked, but they cannot tell you how many of those sales would have happened anyway; only controlled experiments and well-calibrated media mix models estimate that, and always with a range.
Key takeaways
- Attribution divides credit; incrementality tests measure what the ads actually caused.
- Brand search clicks often capture demand that other channels, offline media and word of mouth created.
- Report ad impact as a range with a stated method, and recalibrate models with fresh experiments.
Every ad platform reports how many conversions it drove, and added together those claims often exceed your actual sales. This page explains what each measurement method can and cannot prove, and how to report ad impact to a CFO without overstating it.
What does attribution actually measure?
Attribution splits credit for tracked conversions among tracked touchpoints; it does not measure cause.
A last-click model gives 100% of the credit to the final ad or channel before purchase. Data-driven attribution, which Google Ads uses as the default for most conversion actions, spreads credit based on patterns in your account's conversion paths. Google Analytics 4 now offers only data-driven and two last-click options, after it retired first click, linear, time decay and position-based models in November 2023.
Both only see logged interactions on devices that allowed tracking. A customer who saw a TV spot, heard about you from a friend, then searched your brand looks like a pure search conversion.
Why do brand search numbers look so good?
Brand search ads mostly harvest demand that something else created, so their reported return is the most inflated in most accounts.
People who type your company name already intend to find you, and many would click the organic result if the ad were missing. Display, social, PR and direct mail raise brand search volume, and that halo then shows up as credit for the brand campaign.
Hypothetical example: a brand campaign reports 1,000 conversions a month. During a two-week regional pause, organic brand clicks absorb most of the lost paid clicks and sales barely move. Its incremental value is a small fraction of the reported number.
How do incrementality tests work?
Incrementality tests compare a group that saw the ads with a similar group that did not, which is the only direct way to estimate what the ads caused.
Common designs:
- Geo holdout: pause or cut spend in matched regions and compare sales against regions where spend continued.
- Conversion lift study: the platform randomly withholds ads from a share of the target audience and compares conversion rates.
- Time-based on/off test: alternate spend on and off in blocks, useful when regional splits are impossible.
- View-through checks: test whether impressions without clicks change behavior, rather than trusting the platform's view-through window.
Geo tests need enough regions and volume to beat noise, platform lift studies are run by the media seller, and time tests get distorted by seasonality.
Where does media mix modeling fit?
Media mix modeling estimates each channel's contribution from aggregate spend and sales over time, which makes it useful for budget planning across channels that click tracking cannot see.
MMM needs no user-level tracking, so consent banners do not break it. Google's Meridian is an open-source MMM (Apache 2.0 license) built on Bayesian causal inference. It supports geo-level data and lets you calibrate the model with experiment results and other priors. That calibration matters: an MMM fit on observational data alone can confuse correlation with cause, especially when spend rises in the same weeks demand rises.
| Method | What it answers | Main weakness |
|---|---|---|
| Last-click attribution | Which tracked touch came last | Ignores earlier and offline touches |
| Data-driven attribution | How to share credit across tracked touches | Blind to non-incremental sales |
| Geo holdout | What the ads caused in a test window | Needs volume and matched regions |
| Conversion lift | Causal lift inside one platform | Run by the media seller |
| Media mix modeling | Channel contribution and budget curves | Correlation risk without calibration |
How should you report ad impact honestly?
Give a range, name the method and its confidence, and separate measured lift from modeled estimates.
A good monthly report says something like: "Paid search drove an estimated 300 to 450 incremental orders (geo test, March), versus 700 attributed in the platform." The reader learns the platform number is an upper bound and the test number is the better budget guide.
Practical rules we use in measurement work:
- Label platform-reported conversions as attributed, never as caused.
- Re-run an incrementality test per major channel each year, and feed results into the MMM.
- Use the same margin and cost inputs across channels so ROI comparisons are fair.
Our SEO ROI calculator works from margin and cost, so the output is profit. For organic search, the same discipline applies; how to measure SEO ROI walks through it, and why analytics tools disagree covers the tracking gaps that feed every model.
Attribution is still fine for comparing two ad groups or landing pages inside one account, since the bias hits both sides. The danger starts when attributed numbers are summed across platforms to set the total budget. For a measurement plan that mixes these methods, see our SEO ROI measurement service.
Frequently asked questions
Is data-driven attribution more accurate than last click?
It spreads credit across more of the tracked path, which is usually fairer than last click. It still only sees logged interactions, so it cannot measure sales that would have happened without any ad.
What is a geo holdout test?
You pause or reduce advertising in a set of comparable regions while keeping it running in others, then compare sales. The gap, adjusted for normal regional differences, estimates the incremental effect.
Do small businesses need media mix modeling?
Usually not at first. MMM needs a couple of years of weekly spend and sales data across several channels, so a smaller advertiser often gets more from a simple on/off test on one channel.
Should we turn off brand search ads to test them?
A short, controlled pause in some regions or time blocks is a reasonable test. Watch organic clicks on brand terms too, since some of the paid traffic will move there rather than disappear.