Could ALL the polls be wrong?
I've been looking at the data - and the short answer is yes.
We've had two elections (2015 and 2017) where pretty much every pollster was wrong.
So, what fresh polling disasters await us in 2019? Let's find out...
A thread, with data.
1/15
I've been looking at the data - and the short answer is yes.
We've had two elections (2015 and 2017) where pretty much every pollster was wrong.
So, what fresh polling disasters await us in 2019? Let's find out...
A thread, with data.
1/15
Looking at the data, there are three potential sources of error that could mean all the polls are wrong - AGAIN.
1) 2016 EU referendum weighting
2) Weighting (or not) to 2017 GE vote
3) New voter registrations
Let's explore these one at a time...
2/15
1) 2016 EU referendum weighting
2) Weighting (or not) to 2017 GE vote
3) New voter registrations
Let's explore these one at a time...
2/15
Starting with 2016 EU voting referendum.
All pollsters weight their polls to the 2016 EU referendum result.
They assume that nothing has changed since March 2016 and, if a referendum were held tomorrow, we'd vote 51.9% leave and 48.1% remain again.
This is a problem.
3/15
All pollsters weight their polls to the 2016 EU referendum result.
They assume that nothing has changed since March 2016 and, if a referendum were held tomorrow, we'd vote 51.9% leave and 48.1% remain again.
This is a problem.
3/15
Using freely available government data on births and deaths, we can see what has changed.
From March 2016 to November 19:
1,674,860 people over the age of 65 have died
2,691,333 people have turned 18
What difference does this make? Quite a lot.
4/15
From March 2016 to November 19:
1,674,860 people over the age of 65 have died
2,691,333 people have turned 18
What difference does this make? Quite a lot.
4/15
If we...
1) take this change in demographics
2) apply EU referendum voting intention by age
3) make an appropriate adjustment for age-based turnout
we end up with...
A net loss of 667,494 leavers
A net gain of 882,735 remainers
5/15
1) take this change in demographics
2) apply EU referendum voting intention by age
3) make an appropriate adjustment for age-based turnout
we end up with...
A net loss of 667,494 leavers
A net gain of 882,735 remainers
5/15
This means that, purely through demographic change, the EU vote weightings currently being applied are wrong.
If pollsters are going to weight by EU voting intention, then it should be on the basis of:
Leave 49.6%
Remain 50.4%
Purely based on births and deaths.
6/15
If pollsters are going to weight by EU voting intention, then it should be on the basis of:
Leave 49.6%
Remain 50.4%
Purely based on births and deaths.
6/15
What's the impact of having too many leavers in your polls?
A boost to the Conservative vote share.
If we reduced the 52/48 result used in weightings to 50/50, we'd see the reported Con lead drop by around 3%.
It get's worse though...
7/15
A boost to the Conservative vote share.
If we reduced the 52/48 result used in weightings to 50/50, we'd see the reported Con lead drop by around 3%.
It get's worse though...
7/15
Most EU referendum polls agree that there has been a shift toward remain.
The underlying picture is not the 50/50 you expect with pure demographic change, but 52/48 in favour of remain.
If that is the case, weightings are increasing the Con lead not by 3%, but 6%.
8/15
The underlying picture is not the 50/50 you expect with pure demographic change, but 52/48 in favour of remain.
If that is the case, weightings are increasing the Con lead not by 3%, but 6%.
8/15
Moving on, let's take a look at weighting to 2017 vote.
Some pollsters weight to 2017 GE vote (i.e. they try to ensure 41% of their sample voted Lab in 2017 and 43.5% voted Con).
Some don't. Those who don't consistently show a larger Tory lead.
Why do they do this?
9/15
Some pollsters weight to 2017 GE vote (i.e. they try to ensure 41% of their sample voted Lab in 2017 and 43.5% voted Con).
Some don't. Those who don't consistently show a larger Tory lead.
Why do they do this?
9/15
A study showed people who voted Lab in 2017 are more likely to 'forget' they voted Lab.
But studies have shown that if we regret something, we are more likely to deny it.
When the study was conducted, Lab were lower in the polls, so this isn't forgetfulness, it's denial.
10/15
But studies have shown that if we regret something, we are more likely to deny it.
When the study was conducted, Lab were lower in the polls, so this isn't forgetfulness, it's denial.
10/15
Why does this matter?
Because as Lab have risen in the polls, and people are more committed to an outcome, the denials stop. They suddenly 'remember' they voted Lab in 2017.
The pollsters showing large Con leads don't weight to 2017 GE results. This isn't a coincidence.
11/15
Because as Lab have risen in the polls, and people are more committed to an outcome, the denials stop. They suddenly 'remember' they voted Lab in 2017.
The pollsters showing large Con leads don't weight to 2017 GE results. This isn't a coincidence.
11/15
And now onto voter registration.
People talk a lot about 3m people registering during the campaign. Well that is a fraction of the story.
2019 in total:
18-24: 2531660
25-34: 3088328
35-44: 1694363
45-54: 1073320
55-64: 689353
65-74: 366807
75+ 199595
Total: 9,643,426
12/15
People talk a lot about 3m people registering during the campaign. Well that is a fraction of the story.
2019 in total:
18-24: 2531660
25-34: 3088328
35-44: 1694363
45-54: 1073320
55-64: 689353
65-74: 366807
75+ 199595
Total: 9,643,426
12/15
The electoral commission confirm that an average of 37% of registrations are duplication.
But that still leaves a whopping 6.07m new registrations. Unprecedented.
Of those, 3.53m are aged under 34. A demographic that heavily favours Labour.
What's the impact of this?
13/15
But that still leaves a whopping 6.07m new registrations. Unprecedented.
Of those, 3.53m are aged under 34. A demographic that heavily favours Labour.
What's the impact of this?
13/15
Well, if pollsters are using the most recent official electoral data to weight their polls, they are inflating the Con lead.
As far as I'm aware, only BMG have made a serious attempt to weight accordingly. And they show the smallest Con lead.
14/15
As far as I'm aware, only BMG have made a serious attempt to weight accordingly. And they show the smallest Con lead.
14/15
In concluding, it seems very possible that there are three sources of error which could lead to the polls inflating the Con lead.
In some polls, it could inflate the Con lead by 12%.
There is a very real possibility that this election is much closer than anyone thinks.
END
In some polls, it could inflate the Con lead by 12%.
There is a very real possibility that this election is much closer than anyone thinks.
END
Additional thoughts (1/3):
This thread is getting more attention than I expected - I just want to take a moment to clarify a few points that pollsters have raised.
Firstly - new registrations. Some say this doesn't matter as they simply ask people if they are registered...
This thread is getting more attention than I expected - I just want to take a moment to clarify a few points that pollsters have raised.
Firstly - new registrations. Some say this doesn't matter as they simply ask people if they are registered...
Additional thoughts (2/3):
But it does matter.
Because people who haven't voted before are almost always down-weighted. Their stated votes count for less. This inflated the Con lead.
This also has an impact on EU voting weighting, since previous non-voters are remain-inclined
But it does matter.
Because people who haven't voted before are almost always down-weighted. Their stated votes count for less. This inflated the Con lead.
This also has an impact on EU voting weighting, since previous non-voters are remain-inclined
Additional thoughts (3/3):
I will also say this.
In the 2017 GE on the UK Polling Report forum I identified the turnout weighting error a month before the election.
I provided the data. I modelled the numbers. The pollsters all said I was wrong. We know what happened next.
I will also say this.
In the 2017 GE on the UK Polling Report forum I identified the turnout weighting error a month before the election.
I provided the data. I modelled the numbers. The pollsters all said I was wrong. We know what happened next.
