Thread by @prospect_clark

1/ Yes @YouGov did well last time with their model, but having just spent the evening with a whole bunch of pollsters and political scientists, the interest this one exercise is attracting worries me

2/ as it happens, their projected headline number of Con seats is very close to what I put in the office sweep stake. So I certainly aren’t saying it’s wrong. Only that I don’t know, and neither do they

3/ as the fella who wrote the notorious Guardian front page on Election Day 2015 (headline: “it couldn’t be closer”!) I fear we’re still to absorb the real lessons of that trauma for the pollsters and pundits

4/ Lesson 1 of 2015 is that it’s no use having more data if you’re data is biased. Back then, inspired by the cult of Nate Silver & his state-by-state stuff, we had loads of constituency polls and seat-by-seat projections that turned out to be tosh

5/ by then only oddballs we’re answering landlines for market research: our respondents weren’t typical.

YG has lots of data. Maybe the people who opt in to answering online Qs aren’t so odd. But by definition you can’t correct for any unobserved ways in which they might be odd

6/ generally in polling just now there is heavy reliance being placed on “weighting” - @p_surridge told me earlier she’d seen some raw data recently that showed the two parties neck and neck

7/ now relying on raw data is not a good idea. But there is an opposite risk in weighting and calibrating too much: you end up with your own assumptions, rather than your sample, driving many of your numbers

8/ MRP applies patterns found in big national data to local circumstance. Not a bad idea, but one that doubles down on both the quality of that national data, and the ability of researchers to find & interpret the right patterns in it as well

9/ by making more adjustments, you increase the scope to correct errors for sure, but you also increase the scope to introduce them

10/

And you end up with irresistible seat-by-seat data and projection maps.

A whole lot of precision in other words.

But precision is not the same as accuracy. It can be spurious.

Come to think of it, we had some beautiful pre-election seat maps back in election 2015!

ENDS

PS

11/ if you're into this kind of thing, catch

@StephenDFisher

@martinboon

@p_surridge

@PeterKellner1

@MattSingh_

@chrishanretty

@ProfJaneGreen

@paulhilder

@JohnRentoul

chatting polls, MRP & #GE2019 last night ...
prospectmagazine.co.uk/politics/audio…