For fuck's sake... My aggregate in CA was 60% Keep Newsom if you just allocated undecideds and corrected for whether polls had the right weighting scheme. That's a 2 point error on vote share! Tiny! This is bad for SOME pollsters, not the industry. Jesus gelliottmorris.substack.com/p/polls-of-cal…
https://twitter.com/Wertwhile/status/1441781242467287040
Contrary to popular belief, you don't have to dunk on the polls every time an election result is a little bit surprising to you -- or if you looked at the wrong polling averages
Observing error in a 3rd-party polling average (& 1 that ingested data that was basically withdrawn by a polling house and didn't allocate undecideds!) and then projecting that error onto "polls" as an industry-wide error is a huge analytical misstep IMO.
https://twitter.com/Wertwhile/status/1441901591024570372?s=20
To be clear: There were some really bad polls in California!
But there were also some pretty good polls, especially if you account for undecideds (which you really should in a lopsided state like CA).
But even 3 pts on vote share is a < 1std dev "miss"
But there were also some pretty good polls, especially if you account for undecideds (which you really should in a lopsided state like CA).
But even 3 pts on vote share is a < 1std dev "miss"
https://twitter.com/wccubbison/status/1441902002133299206?s=20
Yes, it’s the obvious answer — but my point is more that the high number of undecideds (which skews the topline) and historical range of error in polls (esp recalls) means the error is not a “failure” for the whole industry, just some pollsters AND a bad aggregator.
https://twitter.com/wertwhile/status/1441903446290878468
Put another way: the result in California is a “failure” for a select group of bad pollsters and those who use outdated methods, and — if analyzed correctly — a pretty good showing for the good pollsters.
The 538 avg muddles this up in damaging ways.
gelliottmorris.substack.com/p/polls-of-cal…
The 538 avg muddles this up in damaging ways.
gelliottmorris.substack.com/p/polls-of-cal…
It is far past time for the most popular public polling analysis website to take seriously the fact that there are some methods and procedures that make polls good and bad and cannot be captured by ad-hoc weighting schemes parametrized based on previous predictive accuracy.
(There is also the matter that there are many unquantifiable “artful” things a pollster can do, eg with question wording, advertising or overall business decisions, that should disqualify their data from entering the public discourse — but that’s another matter)
So, tldr: there are 2 types of errors misleading the conventional wisdom on polls:
1) Analysts are making grave errors in ignoring the things going into the polls, focusing instead almost exclusively on what comes out
2) The public and press expect impossible precise estimates
1) Analysts are making grave errors in ignoring the things going into the polls, focusing instead almost exclusively on what comes out
2) The public and press expect impossible precise estimates
https://twitter.com/gelliottmorris/status/1441901203152162820
• • •
Missing some Tweet in this thread? You can try to
force a refresh
