Thread by @garjoh_canuck

👻👻👻 WHAT PROBLEM DOES GHOST ADS SOLVE? 👻👻👻
This is THREAD #1 about our 2017 JMR paper "Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness” with @EconInformatics & @enub
#MarketingAcad #DigitalMarketing #EconTwitter #AdFX 1/9

Advertisers like Macy’s want to know the ROI of their #advertising campaigns. To understand the #incremental effect of ads, Macy’s need to know what consumers would do *had they not advertised*. Ad experiments deliver these answers. 2/

#Experiments randomly assign users to Treatment or Control groups. Treatment users can see Macy’s ads and other ads too) but Control users can not. In “PSA” experiments, Control users see unrelated ads (e.g. Red Cross public service announcements) in place of the Macy’s ad. 3/

But, PSAs are a real hassle😫. PSAs are also expensive💰: Macy's needs to spend the same per user on PSAs in Control as they do on Macy’s ads in Treatment. So, PSAs don’t scale: they are a barrier to always-on experimentation. 4/

So, what if we show the ’next best’ ad instead of the PSA? Well, now we see the work that PSAs were doing (in the ad logs). Without them, we don’t see🙈 how many Macy’s ads a control users *would have* seen or even if they would have been exposed at all! 5/

Stepping back, advertisers target different types of users. Only some portion of targeted users are actually exposed to Macy's campaign. Ideally, we want to compare exposed users in Treatment to would-be-exposed users in Control. 6/

Problem #2 with PSAs: Modern ad platforms match users to advertisers (e.g. based on click probability). This means the platform will show the Macy’s & Red Cross ads to *different types* of people: Macy’s-exposed & Red Cross-exposed are no longer apples🍎 to apples🍎! 7/

What happens w/out PSAs? We can use the experiment to compare ALL Treatment to ALL Control group users (Intent-to-Treat). This delivers unbiased (🍎to🍎), but noisier🔊 lift estimates.
Why? Users who don’t see ads have 0 ad effect & their data only adds noise to estimates. 8/

In summary, the PSA & ITT approaches have offsetting advantages.
👻Ghost ads👻 (next thread) seek to deliver the best of both worlds🌎🌍: valid✅, precise✅ & scalable✅ experiments.
JMR paper: doi.org/10.1509/jmr.15…
Working paper: ssrn.com/abstract=26200…
9/9

@threadreaderapp unroll