Data journalist & US correspondent @TheEconomist. Author of STRENGTH IN NUMBERS: How Polls Work & Why We Need Them https://t.co/c8nxYdnpks Free newsletter https://t.co/PE6cfXDGQN
7 added to My Authors
Dec 7 • 21 tweets • 5 min read
As always, given complexities with early v in-person votes and geographic variation in partisan loyalty, best to wait for a representative set of each to report before drawing conclusions about these GA numbers. Warnock up big in early vote ≠ victory
For example, the raw partisan split of early votes in Fulton County look a few points better for Warnock than a benchmark for a tied race. But without knowing the election-day numbers in rural areas or urban Atlanta, that’s not enough to make any calls
Dec 6 • 6 tweets • 1 min read
If I had to bet I’d say Warnock has the clear advantage going into tonight’s GA Senate runoff, but that the markets are a bit overconfident on him based on publicly available data. A +2 lead in Senate polls is roughly a 65/35 chance for the leading candidate. Long way from 100
Plenty of other indicators are bullish for Warnock. The same GE fundamentals (fundraising, candidate quality, incumbency) all suggest he ought to have an edge, & we may see the same patterns of GOP voter drop off as in the 2021 runoff. In fact I think that’s likelier than not!
Nov 17 • 6 tweets • 2 min read
trafalgar group has the highest empirical bias relative to other pollster of over 500 firms we have data from. literally the most biased pollster in america. and we warned people that that bias could carry forward and mess with averages in a good year for polls. we were right!!
if someone has a bad record, is opaque about their methods and then tells you they “want to be the elon musk of” that thing, that’s a 🚩 🚩
Nov 11 • 4 tweets • 1 min read
Looking at these newest Senate results and tbh if I’m a news network decision desk, I’m probably getting pretty close to calling AZ for the Dems
and honestly NV is looking good too, but it’s a lot closer and we don’t have super reliable data on the number of outstanding ballots in the counties around Las Vegas and Reno
Nov 10 • 4 tweets • 2 min read
Polls just had one of their best elections ever. Based on our (preliminary) estimates, our averages in competitive states have an absolute error of just 2 points on margin — a little under half the expected error — with average *bias* undershooting Dems.
This chart shows bias in each state in 2022 vs 2020. Take out FL and there is a uniform decrease in bias across states. Good!
Though if you're looking for reasons to be anxious: much like 2018 there is still a positive slope here, indicating *some* persistent partisan nonresponse
Nov 10 • 4 tweets • 2 min read
this is a completely useless exercise if you’re not imputing results in uncontested seats. you do not capture structural advantages when aggregate totals are missing like 50 seat observations!
If you adjust for the percent of votes outstanding in each state in Brazil right now*, you get a result of 50.1 Lula - 49.9 Bolsonaro --
which is to say, it looks like Lula could still pull this off, but it's going to be another close round!
The * here is that doing this assumes that state voting patterns will be constant as new votes come in, which is not true. So it's an estimate with a wide margin of error!
Oct 28 • 4 tweets • 2 min read
gun to my head i would bet that polls are overestimating democrats again, rather than republicans. but such predictions don’t have a great historical track record…
still the clues here are mostly constant across polls and have helped anticipate recent errors (low n tho)
remember, registered republicans were more likely than democrats to answer polls for much of 2020, including during the summer boost in response rates from Dem activists — so even well designed polls that stratify &/or weight by party don’t solve nonresponse by vote intention!
Oct 24 • 6 tweets • 3 min read
How do we deal with partisan polls? Two options:
1) Let the polling aggregation models make adjustments for whether a poll was conducted for/released by a partisan client (this means subtracting ~3-5 pts on margin from that side)
So while the claim is that GOP partisan polls are pushing up Republican numbers in the averages, what we see from these two model runs is when you exclude those partisan polls from the averages Democrats actually end up _worse off_ in most states
Oct 24 • 4 tweets • 2 min read
Really good polling journalism from @qdbuinytimes.com/interactive/20…
The "true" interval is probably EVEN LARGER due to coverage error (that's when a population isn't represented by the sampling frame, such as "ppl with landlines" or "X online panel"), processing error (are the targets right? do they control for non-response?) or measurement error
Oct 20 • 24 tweets • 6 min read
You may have noticed some peculiar Senate polls from a "nonpartisan" group called Center Street PAC this cycle. Polls like these, where Democrats are doing 10-20 points better than the averages.
I interviewed their pollster and got my hands on their data. Here is what I found:🧵
Here is the article if you want to skip the gory details: economist.com/united-states/…
The first thing to do is establish whether CSP is an outlier. So we made a model that measures how far each pollster's data deviates from averages, accounting for partisan bias & a few other vars.
Oct 17 • 6 tweets • 2 min read
hey siri what is the margin of error on a sample size of 77 people
(and that’s not even taking non-sampling error into account)
Sep 16 • 4 tweets • 2 min read
Bias in polls has not been predictable nationally (see 538)
But this ignores that _patterns in bias across states_ may be more predictable now (see tweets)
The Q is not so much "will polls be biased nationally" but "will polls be biased in the close states with key elections"
FWIW I do not think that anyone will have a great record trying to predict the precise degree of bias in the polls (partly because of the small sample size of elections). But I do buy the theory that the probability of polls overestimating Dems is higher than overestimating Reps.
Sep 1 • 4 tweets • 2 min read
Dems +3 in the AK-AL special beats our benchmark based on fundamentals by 14 points.
That pushes our forecast of the House pop vote based on patterns in past specials to 51.5%. If it sticks, that'll be the best performance for an incumbent party since 1990 (as far as our data go)
there are some fair Qs about how predictive of Nov AK-AL can be. specials are idiosyncratic, and voters might not be representative of midterm LVs. + the AK electoral system is also unique & poli psych might not translate elsewhere.
still… big swing, similar to other specials
Aug 30 • 8 tweets • 3 min read
Some important polling simulations here:
If you believe Senate polls today have no national bias toward the Democrats, then they are ~80% to hold the majority (based only on polls, not fundamentals)
If you think polls will be as biased as in 2020, they are slight underdogs 1/2
Consider state biases and the picture is bleaker
Above, I'm simulating national polling bias (eg equal bias across all states) then adding state-lvl err
But Ds this yr are susceptible to the same (potential) bias in the northern battlegrounds as in 2016/20
Accounting for that:
Aug 24 • 7 tweets • 2 min read
So, based on our not-yet-released election model, special elections so far this cycle are consistent with Democrats winning 51.3%* of the two-party vote for the House in Nov. That is almost exactly the threshold they need to keep their majority.
Big increase after NY-19 and NY-23
*Caveats: (1) We don't have final totals in the AK-AL special yet, so that's not in our model. (2) There is time for the metric to change before election day, this is just where things are now. (3) The special election forecast is not perfect, MOE is a bit over +/- 2.3pts today.
Aug 18 • 4 tweets • 2 min read
absolutely huge margin of error on senate polls taken this far ahead of elections
Jul 21 • 4 tweets • 2 min read
Lots of pollsters aren't doing what is TBH the bare minimum at this point to even try to adjust for partisan nonresponse outright or disclose whether they detect it in their data. This is perhaps the biggest story with polls right now — and most are just shrugging their shoulders
To be clear, this is not an easy problem to solve! (I report on some potential improvements in chapters 6 & 7 of gelliottmorris.com/strength_in_nu….) But I don't think consumers of polls really know how big the effects of residual partisan biases in samples are right now.
Jun 30 • 4 tweets • 2 min read
Do you remember an article from earlier this week that claimed one million voters switched to the GOP last year? Turns out it was completely wrong. The authors mistook modeled party ID scores for actual party registration. Political numeracy matters!
I do not think this is right. If the QPac poll is suffering from partisan nonresponse, that means it has a group of very engaged and liberal Dems answering it. Those are also the Dems most likely to disapprove of Biden right now.
NEW The Economist/YouGov poll fielded after SCOTUS struck down Roe v Wade
- 49% of adults disapprove of the decision (42% approve)
- 57% support Congress legalizing abortion in the 1st trimester (32 oppose)
- 53% support a federal law ensuring a right to an abortion (37 oppose)
- 5% of adults say abortion should be banned always
- 36% call themselves "pro-choice" v 25% "pro-life. "Pro-choice" is up from 32% last week.
- 47% say they would vote "for or against a candidate just on the basis of their position on the abortion issue?"