Thread by @gelliottmorris

In all seriousness, this tweet from @SeanTrende exemplifies a very important difference in how various poll aggregators view actual polling microdata — one that has the potential to reshape how news consumers view political polls writ large (for the better, IMO). Short thread:

Some polling aggregators (eg RCP) take the raw polling toplines data and assume that enough pollsters will follow best practices in weighting, sampling, etc to give you the best average prediction possible. This serves us well sometimes but is pretty naive when you drill down.

Others (eg 538) think that there are consistent differences between pollster modes, populations and polling firms that allow you to makes good predictions of which methods are best. They weight polls by accuracy and do other math to debias data to squeeze out all the extra juice.

And then there are those of us (including myself) that view polls with an eye toward their internal design and modeling. We think that critiquing crosstabs and methods is a good way of figuring out *why* some polls are better than others. In applied modeling, this means stuff

like adding corrections for which variables pollsters weight on. Instead of saying there are poll/topline-level characteristics that we can adjust for, we are saying there are crosstab-level differences — & incorporating them adds value to our averages that other approaches miss.

One ex of this is the Georgetown Battleground Poll, which earlier this year had a college-educated sample near 60% IIRC. Some aggregators just threw it in the average (bad). Some adjusted for historical accuracy (better). And some of us took the raw data and reweighted it based

on our own predictions of the racial and educational makeup of the electorate.

That is all indicative of, IMO, and important development in poll aggregation. It's not enough to just look at toplines and historical accuracy. We can use design-level info to improve our models.