Thread by @Richard_Ampere

Could the Vote Leave overspend have swung the EU referendum?
- A thread combining #Brexit and #AdvertisingEffectiveness analysis – conducted in a personal capacity.

Here goes…

Thanks to the work of journalists like @carolecadwalla , the @CommonsCMS Fake News investigation and @ElectoralCommUK, we now have more insight into the advertising metrics from campaigns run on Facebook during the EU referendum on behalf of Vote Leave

The @ElectoralCommUK found that Vote Leave “worked to a common plan” with BeLeave and thus exceeded its statutory spending limit -
electoralcommission.org.uk/__data/assets/…

The full effect of the money spent on Facebook advertising on behalf of BeLeave will unfortunately never be known, but is it actually plausible that it could have swung the result of the vote?

Let’s examine the facts...

DCMS documents parliament.uk/documents/comm… list advertising run by AIQ on behalf of Vote Leave and BeLeave.

The exact number of advertising impressions isn’t revealed, but crunching the min/max figures indicates that across the sets of data published by DCMS, the adverts were seen between ~150 million and ~350 million times in aggregate by UK Facebook users.

BeLeave advertising accounts for an average of ~26% of this volume – and potentially as much as ~43%, depending how the actual impression data fell.

At 51.8% vote share, Leave’s winning margin in the vote was 1.2 million – so would have been enabled by a ~600,000 vote swing from Remain.

The change in the run-up to the referendum, based on polling averages, was as much as 6-8 percentage points towards Leave – between 2 million to 2.7 million votes. It was during this period that Leave/BeLeave’s Facebook ad campaign was run. Also see: whatukthinks.org/eu/questions/i…

Given that somewhere in the region of 33 million to 37 million people use Facebook in the UK, and as many as 350 million adverts were delivered, it’s certainly highly possible that 600,000 people, or 1.6%-1.8% of Facebook’s users, were influenced by Leave’s Facebook advertising.

Specifically however, could the spending on behalf of BeLeave *alone* have accrued the necessary 600,000 votes? This depends partly on how much of the pre-referendum swing is attributable to the Facebook ad campaign.

Here are two possible scenarios under which this would have happened:

i) BeLeave accounts for 26% (the average) of the Leave/BeLeave advertising impressions, and the overall campaign/s delivered just over a 6% swing in voting intention.

ii) BeLeave accounts for 43% (the max possible) of the Leave/BeLeave advertising impressions and the overall campaign/s delivered just over a 4% swing in voting intention.

Other combinations of the above could also have achieved comparable results.

Scenarios with lower swings attributable to the Facebook ad campaign would likely have meant that the BeLeave advertising spending alone did not change the final referendum result.

So the question now becomes whether it is plausible to attribute such a large voting intention swing towards Facebook advertising alone?

Commercial sector benchmarks (from advertisers & agencies), indicate that anywhere from 0.5%-1.6% of people typically click on a Facebook advert, and a further 5%-15% go on to make a purchase.

In this case, a 4% swing attributable to the campaigns would be running at ~3-4x commercial sector conversion benchmarks, so would be high compared to these commercial metrics...

...But, it should be noted that these benchmarks are typically based on ads persuading people to buy something, not persuading people to vote. In addition, campaign effectiveness is highly variable, so even in the commercial sector such conversion rates would not be unheard of.

In summary, while most definitely far from certain, it does appear at least *possible* that the overspend alone could have swung the referendum result.

The caveats. The analysis assumes that the DCMS documents detail all of the adverts delivered as part of the Facebook campaigns, and that I have extracted from the pdf format correctly (it was not the cleanest dataset in the world)...

...It also assumes that the adverts targeted on behalf of VL and BL were equally effective in their impact. For simplicity, the scenarios look only at swings from Remain to Leave, and do not incorporate non-voters becoming voters.

Postscript:

Fun fact 1. Under the scenarios explored in this thread, it takes between 125-200 advertising impressions to switch a vote.

Fun fact 2. Switching a vote under the same scenarios would therefore have cost in the region of £2-£3.