Nate Silver Profile picture
Editor-in-Chief, @FiveThirtyEight. Author, The Signal and the Noise ( Sports/politics/food geek.
C. P. H. Profile picture Judy Griffin Profile picture Rosalind Stewart Profile picture eDo Profile picture Michael Platt Profile picture 270 added to My Authors
10 Aug
I think the incentives toward accuracy are pretty decent. It's easy to compare polls to one another and also to compare them against the actual result. And pollsters make a lot of claims about their accuracy when soliciting new business. Plus there's professional pride, etc.
You do encounter issues that also occur elsewhere in political media. For many outlets, most of the audience will consist of political partisans. Even for these outlets, though, polling is sometimes an avenue they use to bolster their claims of unbiasedness and objectivity.
For example, whatever else you think about Fox News, their polls are both very good and quite unbiased (if anything they've had a tiny D bias in recent elections).…
Read 5 tweets
9 Aug
Something else to keep in mind is that presidential elections tend to tighten down the stretch run. So an 8-point Biden lead today would translate into around a 6-point popular vote victory on Election Day, on average.
Of course, you can use a prior other than zero based on the economy or incumbency or what not if you want. Given recent improvement in economic data + high polarization, the prior our model will use is currently something like Biden +2, which is not so different than zero.
But the empirically-driven intuitions that (1) elections tend to be close in a highly polarized era and (2) Trump will tend to win most of the close elections because of the Electoral College, are things that should keep people from being too overconfident about Biden's chances.
Read 3 tweets
4 Aug
Like all folks modeling presidential elections, we're forced to use small samples (e.g. n=12) to infer what the broader universe (n=∞) looks like. This is hard! But if you overfit models to past data, you'll often make poor inferences about what unobserved data looks like.
So while we spend a lot of time thinking thru different model specifications, it's mostly *not* about trying to maximize fit on past data. It's more about thinking through edge cases and specifications that will be robust under a variety of conditions.
The reason we generally prefer polls>economy is because different modeling choices on the economy can yield radically different answers (i.e. a Biden landslide if you look at GDP or a Trump landslide if you look at income) while different polling averages produce similar answers.
Read 3 tweets
30 Jul
Offered as an illustration of the limits of "fundamentals"-based election models:

The "Time for Change model", as specified here, predicts that Trump will lose the popular vote by ~36 points (not a typo) based on that 2Q GDP print.…
On the other hand, if you ground your model in Real Disposable Income like some others do (e.g. the "Bread and Peace" model) you may have Trump winning in an epic landslide. Possibly the same if you use 3Q GDP (forecasted to be +15% annualized) instead of 2Q.
It certainly helps to use a wider array of indicators over a longer time frame (that's what we do) but the notion that you can make a highly *precise* forecast from fundamentals alone in *this* economy just doesn't pass the smell test.
Read 7 tweets
29 Jul
I have a few thoughts on this and similar polls that ask people if they'll get vaccinated.

1. In the medium run, the limiting factor is likely to be access and availability to the vaccine.
That is to say, if we get to the point where **everyone who wants a vaccine can get one in reasonably short order**, then we can start thinking about how to persuade more people to be vaccinated. But getting to that first point may not be easy.
2. Headlines on polls like these can sometimes be misleading by treating "not sure" responses as "no". You'll have a poll result like e.g. 40% yes, 20% no, 40% "need to learn more", and the headline will frame it as "only 40% will get vaccinated!" when really it's 40-80%.
Read 8 tweets
29 Jul
At the end of the day, it's only July, there are a lot of unprecedented events happening, polling is more accurate than its critics think but probably *not* as accurate as it was in say 2004-2012—and if we wind up in a photo finish, Trump likely wins b/c of the Electoral College.
So while Biden has a robust polling lead now, and a number of other things going for him, it's a little early to be all that confident about what is going to happen.
I think part of it, too, is the pandemic warps people's sense of time (it certainly warps mine). November feels weirdly soon if you don't have the usual milestones and rhythms to ground yourself. But in political time, the final 99 days of the campaign are often an eternity.
Read 3 tweets
28 Jul
FWIW, here's a regression-based estimate of Biden's current standing in each state. This works by taking our state polling averages, and trying to figure out what differences there are from past elections based on regional or demographic variables (without overfitting the model).
As you can see, this matches polling data well. Then again, it's supposed to do so. In fact one limitation of the this technique is that if there are biases in the polling (say, they underestimate Trump's standing with a certain group) they'll also be reflected in your estimates.
This ^^ is also an issue for techniques like MRP (in fact, even more so, since MRP generally uses just one pollster's data). Still, it's fun to look at and does have *some* predictive accuracy. Among other things, clear that CO and VA are pretty solidly into blue territory now.
Read 4 tweets
27 Jul
I wish there was more explicit acknowledgment that one of the most persuasive rationales for continued distancing/lockdowns is indeed to buy time for technological improvements. Not just vaccines but also therapeutics, rapid tests, greater scientific understanding.
Without those technological improvements, then either lockdowns just *postpone* the inevitable (since no matter how diligent your lockdown, cases will start increasing again one you relax it) *or* you're basically arguing for indefinite lockdowns.
Read 2 tweets
27 Jul
Basically, the additional uncertainty introduced into an election forecast by COVID-19 falls into two buckets:

1) It means there's a lot of *news* and economic volatility.

2) It could screw with the mechanics of voting, vote-counting or polling in unpredictable ways.
Bucket 1) is easy enough to handle empirically. You can look at whether election cycles with lots of major news developments tend to be associated with more polling volatility and/or polling error, and indeed they do, although note the usual issues with small sample sizes.
Bucket 2) is a lot trickier, I think, since there are no real solid precedents for COVID. (Maybe something like Hurricane Sandy, but that only affected a couple of states.) But there have been too many issues IMO with voting in these late-stage primaries to ignore it.
Read 9 tweets
23 Jul
As an FYI, there have been a fairly high number of polls with partisan sponsors in the presidential race this year. (Usually these are more common in races for Congress.) We DO include these polls in our averages, which is a change from the past, but with an adjustment.
Namely, we use a stricter house effects adjustment; the model assumes that these polls are biased until proven otherwise. And it weights them less.

However, for boring technical reasons, the model wasn't catching all the partisan polls and adjusting them like it was supposed to.
So as an FYI, we fixed that tonight, which slightly changed the averages in some states. Also, here is our newly-updated polls policy, which explains what is considered a partisan poll.…
Read 5 tweets
22 Jul
Obviously, any poll showing the Democrat up in Texas is a pretty good result for the Democrat, but if Quinnipiac has Biden up 1 in Texas when he's up 8-9 nationally (and more than that in Q-Pac's last national poll), it's nowhere near the tipping point.…
One thing that I think people miss is that the partisan lean of a state is slightly mean reverting.

Say a state was an R+20 state eight years ago, and an R+10 state four years ago. You might expect it to be a swing state now, right? No. On average it'll be like an R+13.
And I'd argue that you're seeing this in the data this year. Biden is doing "surprisingly" well in the Midwestern states like OH, MI and WI that drifted Republican from 2012 to 2016. But his numbers have been pretty tepid in AZ, GA, TX, states where Dems were gaining ground.
Read 5 tweets
19 Jul
Good chance deaths will increase further. But it does seem likely that the death *rate* from COVID is decreasing, which is good news, even if the number of deaths is increasing because we have so many freakin' cases.
One thing I like to look at is the lagged case fatality rate (CFR), which is the number of deaths divided by the number of cases some time ago.

Over the past week, the lagged CFR has been 1.5% using a 2-week lag. If you prefer a 3-week lag, it's 1.9%.
Of course, because of mediocre testing, we're still only catching a small-ish share of infections, by most estimates. If, per @youyanggu's model, we're catching one-fifth to one-sixth of cases, that means the *infection* fatality rate (IFR) is somewhere in the range of 0.3-0.4%.
Read 4 tweets
17 Jul
Thread. The US never really decided between suppression (keep R<1) and mitigation (take limited measures to e.g. keep hospitals from overflowing; basically, Sweden) and these regionally-driven flare-ups are the somewhat inevitable result.
We've seen that states and people *will* change their behavior when cases are noticeably increasing (say R >= 1.2 or so). But, there's a lag before that improves the situation, during which time things can get pretty out of hand.
Read 2 tweets
17 Jul
THREAD: In forecasting Trump's fate, one needs to be careful as there is a fair amount of uncertainty as to what the trajectory of the pandemic will look like by November. Overly precise predictions of COVID numbers in X country/state by Y date have generally not fared well.
What we can say for sure is that the US has *already been hit really hard*. And even if the numbers are improving by the fall—which, to be clear, may take some luck—we'd probably be looking at spring before there's a robust economic recovery and things feel 85% "normal" again.
But there probably is a narrow window where by Nov., there have been ~2 months of real improvement. Would that be enough to reelect Trump? I don't know. I tend to think the public would still give him bad marks for COVID, and keep in mind he was behind before any of this started.
Read 5 tweets
15 Jul
Be careful about premature victory laps.

A lot of countries that had once seemed to have COVID under control are now having issues. Israel is the one where things have spun very out of control. Others here may be more containable.
In a slightly different category are middle-income countries like India and South Africa that initially imposed harsh lockdowns when they had low caseloads (i.e. there was no real first wave) but relaxed those lockdowns and now aren't doing all that much to fight back.
I don't mean to be a downer. But I think The Discourse might be improved by acknowledging fighting this thing requires diligence, patience, good governance, great science, and maybe a bit of luck.

And where good governance is lacking (🇺🇸), you need extra of the other qualities.
Read 3 tweets
14 Jul
If you read through Florida's very detailed reporting on test results by clinic, the clinics that aren't reporting negative tests (or report very few) are mostly quite small and look as though they represent around ~3% of the positive tests in Florida.…
This likely does *not* affect the statewide numbers that much, in other words.

And there's probably degree of this going on in every state. With 100's of test providers, some are going to report incomplete or anomalous data.
Read 2 tweets
14 Jul
So, Gravis Marketing has released 5 polls of the presidential race in the past several weeks. Three were for OAN (One America News) and three weren't. Here's what they showed:

Not for OAN
FL: Biden +10
MN: Biden +16

NC: Trump +3
GA: Trump +3
AZ: Trump +4
I know it's just 5 polls. But the three polls for OAN are literally some of Trump's best polls during this time period and the two non-OAN polls are literally some of his worst polls. The sponsor shouldn't really have an effect on poll results like that.
That other Florida poll didn't meet our standards for inclusion (and we have inclusive standards) because of the...weirdness in how OAN disclosed the results.

But a TIE in a Gravis/OAN Florida poll vs. Biden +10 in a non-OAN Gravis poll fits the pattern.
Read 3 tweets
13 Jul
US daily numbers via @COVID19Tracking:

Newly reported deaths
Today: 327
Yesterday: 476
One week ago (7/6): 242

Newly reported cases
T: 58K
Y: 61K
7/6: 47K

Newly reported tests
T: 721K
Y: 727K
7/6: 520K

Positive test rate
T: 8.1%
Y: 8.4%
7/6: 9.1%
This is all very bad relative to what the data looked like a few weeks ago.

It is also slightly less bad relative to some recent days.

But Mondays (which mostly reflect what happened on Sunday) are often pretty slow.

Good that the positive test rate has flattened out a bit.
Among the "terrible trio": Arizona does seem to show some flattening, which makes sense since it was the earliest of the three to spike. And positive test rates have fallen in Florida, although they're reporting *lots* of cases (and tests). Jury is still out on Texas.
Read 4 tweets
13 Jul
I read a lot about COVID-19 and immunity and if your priors aren't something like "there's immunity in the short run and then it's complicated and uncertain in the long run" then I'd suggest you read more about it.

A lot of headlines and tweets about this subject are misleading.
Also, be wary of anecdotal stories about people getting "re-infected". Some may be true. Some may be false positives or people that were sick the whole time. COVID is weird; the immune system is weird. Things that *can* happen aren't necessarily *common*.
To illustrate the point: a prediction market estimates that 99.7% of people who had COVID will still be immune after 6 months. If that were true (have no idea) would be pretty good news but you'd ALSO expect to see reports of reinfection among the 0.3%.
Read 3 tweets
13 Jul
Which states have seen the largest increase in the nonwhite share of their population since 2010?

NV +5.9
CT +5.3
ND +5.2
MA +5.0
WA +5.0
RI +5.0
MD +4.7
NJ +4.7
FL +4.7
TX +4.1
MN +4.0
GA +3.9
AK +3.9
NE +3.9
PA +3.8
Observations: Not sure people realize the extent to which the Northeast is becoming more diverse (except Maine/NH/VT which rank toward the bottom of the list).

MN is diversifying more quickly than WI and MI, which might hedge against it becoming a tipping-point state like WI/MI.
Colorado isn't diversifying as much as people think. Actually ranks below average.

No state became whiter between 2010 and 2019, although Washington DC did.

Among actual states, South Carolina saw the smallest increase in its nonwhite population share.
Read 4 tweets
12 Jul
US daily numbers via @COVID19Tracking:

Newly reported deaths
Today: 476
Yesterday: 757
One week ago (7/5): 209

Newly reported cases
T: 61K
Y: 63K
7/5: 43K

Newly reported tests
T: 729K
Y: 634K
7/5: 657K

Positive test rate
T: 8.4%
Y: 9.9%
7/5: 6.5%
While the number of deaths might seem comparatively low, Sundays are usually slow for reporting, and in fact this is the most deaths reported on a Sunday since 5/31. So, pretty bad news. With that said, we're still learning about the reporting patterns of the current hotspots.
There were quite a lot of tests reported today, particularly in Florida, and in some states, the positive test rate seems to be plateauing.

But, so far optimism about plateaus has not "aged very well"; you'll think you've hit one, then you get some worse numbers the next day.
Read 4 tweets