Without wading into this specific debate, the feud over MRP gets at a pretty fundamental question about survey research right now:
do polls work because they have a design that yields a representative sample, or do polls work because they have a good model to correct error?
do polls work because they have a design that yields a representative sample, or do polls work because they have a good model to correct error?
The design-based view is the classical view. Here, a probability sample = a good poll. If everyone has a known chance of selection, then the results should fall within a theoretical margin of error due to sampling
The model-based view says design doesn't do it anymore; that's why they all use extensive weighting.
Instead, polls work because of the model: we basically just make inferences about the population based on an unrepresentative sample by controlling for demog. used for weighting
Instead, polls work because of the model: we basically just make inferences about the population based on an unrepresentative sample by controlling for demog. used for weighting
The truth is probably in between. A good design substantially reduces the burden on your model; a good design does not obviate the need for a model.
But using MRP to make an estimate for your poll, rather than weight a survey, is a logical extension of the model-based view
But using MRP to make an estimate for your poll, rather than weight a survey, is a logical extension of the model-based view
In the model-based view, rake-weighting is just a really horrible form of modeling; may as well use the good kind we'd use in any other contexts.
But to some, it's basically an abandonment of the idea that it's a survey at all.
But to some, it's basically an abandonment of the idea that it's a survey at all.
For a poll aggregator, there are some additional consequences. Poll aggregation makes most sense in a design based world: if a good design yields unbiased samples around an average, a robust average should be very good
In a model-based world, poll aggregation gets tricky and the incentives don't line up. An individual pollster would make a very different choice on bias-variance tradeoff on individual polls, for instance, than a poll aggregator
(or, i should say, an individual pollster might make a very different choice on bias-variance trade off in model selection then a poll aggregator would want that pollster to make)
And some kinds of model-based polling estimates--like those that pool data across states or over time--could be even harder to treat like a typical poll
Either way, some of the systematic polling errors in recent years, or polls that look like (or may be) 'herding', is more like what you would expect in a model-based survey world than a design-based one
Which is to say: whether you draw the line at MRP or not, polling averages are probably already suffering some of the costs of a model-based polling world
