Here's a first look at the preliminary research into the 2020 general election polling error.
So far, it's most definitive in ruling out potential causes -- e.g. education weighting, late vote shift -- that *don't* seem to have been major factors.
This is no surprise if you've been following discussions, but focus is coalescing around some form of differential non-response - that is, people who were polled are different from people who weren't, in a way current weighting didn't account for.
For instance, even if a poll included the correct number of Republicans compared to the actual electorate, Republicans in the poll might have been less supportive of Trump than Republicans who weren't reached.
These are preliminary findings -- the AAPOR task force (which, disclosure, I was a part of) is expected to release a full report soon.
In the meantime, important as always to note both that 1) this was a significant error that deserves a lot of further work and 2) polling in general is still not useless or broken beyond repair. See, e.g. recent surveys on vaccines, which line up w/ official data.
Even at its absolute best, polling is always going to give you estimates, not precision. But if you understand those limitations, it can still be a valuable tool for understanding the people around you at a macro level.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
I've information anecdotal, chemical, and clinical
I know the COVID experts, and I quote-tweet fights on aerosols
List BioNTech to Zeneca, in order categorical
I'm very well acquainted, too, with matters virological
I understand most models but I'm baffled by the cubical
About t-cell immunity I am teeming with a lot o' news...lot o' news...
with many cheerful facts about the antibodies I'll produce!
I know the vaccine history from Jenner and the old cowpox
I answer diagnostics, I thank all the nurses and the docs
I quote in elegiacs all the CDC analysis
And hope we'll usher in an annus slightly more mirabilis
I can tell adenoviruses from mRNA and spike proteins
Secondary question now is probably whether decreased confidence in J&J actually spills over into decreased vaccine willingness, or just results in more people having a preference for an alternative vaccine.
New from me: When the first COVID-19 vaccine was authorized last December, a lot of Americans were on the fence about getting it. Since then, there's been a dramatic shift toward vaccine acceptance -- but the remaining holdouts could be more stubborn.
Plus, some lessons from vaccine surveys that apply to polling in general:
-Snapshots, not predictions
-People are bad at predicting future behavior
-Question framing matters
-Look at the undecideds!
This is yet another polling story where the presence/absence of some sort of "not sure" option is crucial. Polls that didn't have an explicit "not sure" option showed higher initial support for vaccination than ones that did, and less substantial movement over the next few months
This is excellent work, and I'm so glad someone tested this out.
"Telling people that Mr. Biden and Mr. Trump have been vaccinated...has an impact on some but not others...our findings suggest that presidents can nudge some of them further along that path."
"Republicans who outright say 'no' do not budge when told of [Trump's] endorsement of vaccines; however, among Republicans more on the fence, we see movement."
Also, recontact data: "Among those who previously said they were planning to get vaccinated, 16% now report having received at least one dose in our survey's control condition. Moreover, one in five of those who previously said maybe to getting a vaccine now say yes outright."