YouGov MRP turnout weighting. Oh dear.
I posted a quick analysis thread last night, and raised concerns about the turnout weighting.
In this thread, let's explore the issue further.
I'll try to give a simple, clear explanation of the problem.
1/13
I posted a quick analysis thread last night, and raised concerns about the turnout weighting.
In this thread, let's explore the issue further.
I'll try to give a simple, clear explanation of the problem.
1/13
Firstly, how do polls measure turnout (i.e. % who turn up and vote)?
After 2015, most except Kantar & YouGov MRP use 'self report'.
i.e. they believe you if you say you are 100% gonna vote.
They may however 'downweight' you if you are a first time or previous non-voter.
2/13
After 2015, most except Kantar & YouGov MRP use 'self report'.
i.e. they believe you if you say you are 100% gonna vote.
They may however 'downweight' you if you are a first time or previous non-voter.
2/13
'Downweighting' refers to taking your vote, and reducing it's value by a percentage.
For first time/previous non-voters (DNV), this may be 50%. This is based on historical voting patterns.
But what happens if the dynamics change, and historical patterns don't hold?
3/13
For first time/previous non-voters (DNV), this may be 50%. This is based on historical voting patterns.
But what happens if the dynamics change, and historical patterns don't hold?
3/13
What if, like now, there is a measurable increase in political engagement?
Then for pollsters who downweight first timers/DNV's, it means they will underestimate the impact these voters have at the polls.
These voters favour Lab so it reduces Labour vote % in the polls.
4/13
Then for pollsters who downweight first timers/DNV's, it means they will underestimate the impact these voters have at the polls.
These voters favour Lab so it reduces Labour vote % in the polls.
4/13
This is one of my potential 'three sources of error' for the 2019 polls, identified previously.
And if the dynamics have changed, then down-weighting first-timers/DNV's is bad enough on it's own.
But the YouGov MRP goes much, much further down the rabbit hole.
5/13
And if the dynamics have changed, then down-weighting first-timers/DNV's is bad enough on it's own.
But the YouGov MRP goes much, much further down the rabbit hole.
5/13
The YouGov MRP does not ask people how likely they are to vote AT ALL.
It makes an assumption based on turnout from, essentially, the 2015 GE.
This assumption is that youth turnout will be 52%. No ifs, buts or maybes. The MRP has ordained it.
And it's based on a fallacy.
6/13
It makes an assumption based on turnout from, essentially, the 2015 GE.
This assumption is that youth turnout will be 52%. No ifs, buts or maybes. The MRP has ordained it.
And it's based on a fallacy.
6/13
The fallacy it's based on is this:
"the turnout of 18-30's didn't change much between 2015 and 2017 elections" (I'm paraphrasing)
This is based on a BES study that is very much contested.
Other credible studies have reached the opposite conclusion.
7/13
"the turnout of 18-30's didn't change much between 2015 and 2017 elections" (I'm paraphrasing)
This is based on a BES study that is very much contested.
Other credible studies have reached the opposite conclusion.
7/13
They also ignore real-world evidence.
The polls in 2017 that did assume turnout to be at 2015 levels were catastrophically wrong.
ICM, BMG, ComRes, Panelbase all used this turnout model and all hugely underestimated Lab %.
Survation used self-report and called it right.
8/13
The polls in 2017 that did assume turnout to be at 2015 levels were catastrophically wrong.
ICM, BMG, ComRes, Panelbase all used this turnout model and all hugely underestimated Lab %.
Survation used self-report and called it right.
8/13
The idea that the turnout models were not to blame for the polling mess of 2017 is not a credible position to take.
It's a mixture of denial and strained post-rationalisation that misses the blindingly obvious.
It's seems likely the BES didn't get this one right.
9/13
It's a mixture of denial and strained post-rationalisation that misses the blindingly obvious.
It's seems likely the BES didn't get this one right.
9/13
But, even if the BES did get it right, there is a bigger question to answer:
Why would you build an assumption into your model that 'nothing has changed'?
How on earth could you know? What changes in dynamics and behaviour might you miss?
What if things are different?
10/13
Why would you build an assumption into your model that 'nothing has changed'?
How on earth could you know? What changes in dynamics and behaviour might you miss?
What if things are different?
10/13
It doesn't really matter if you do think things are different from the 2015 GE or not (I think most agree they are)
The question is - COULD such changes occur?
If the answer is yes, why use a turnout model that ignores potential changes in demographically based turnout?
11/13
The question is - COULD such changes occur?
If the answer is yes, why use a turnout model that ignores potential changes in demographically based turnout?
11/13
This isn't to say the YouGov MRP turnout weightings will be wrong
"even a stopped clock is right twice a day"
But if it's right, it will be because of a healthy dollop of luck.
In a political environment where everything HAS changed, this is a high risk approach.
12/13
"even a stopped clock is right twice a day"
But if it's right, it will be because of a healthy dollop of luck.
In a political environment where everything HAS changed, this is a high risk approach.
12/13
In conclusion, with an estimated 3m new voters registered since GE17, record political engagement, and an election dynamic like no other, the turnout model could be the MRP's achilles heel.
The impact of this could well be a significant underestimation of Lab vote %.
END
The impact of this could well be a significant underestimation of Lab vote %.
END
@abandonnuance To clarify what this means
1) YouGov MRP is using GE15 turnout figures
2) If in this election, we see GE17 turnout figures, the YouGov MRP is definitely wrong - no ifs, no buts
A rough calculation would be a change to ~36-42 vote share and well into a hung parliament
REAL END
1) YouGov MRP is using GE15 turnout figures
2) If in this election, we see GE17 turnout figures, the YouGov MRP is definitely wrong - no ifs, no buts
A rough calculation would be a change to ~36-42 vote share and well into a hung parliament
REAL END
