The problem with 538's definition of "polling error" goes back to experimental design
That is, does this thing measure what we're saying it does?
In the case of "poll margin - election margin = poll error" they're assuming poll attempts to measure the final outcome. It doesn't!
That is, does this thing measure what we're saying it does?
In the case of "poll margin - election margin = poll error" they're assuming poll attempts to measure the final outcome. It doesn't!
If you're not a science person, and you're like "what does this mean?" Here's how I break it down
Polls measure *preferences* of decided voters, and *how many* undecided voters. That's it!
Elections, unlike polls, don't include undecideds. This means variables have changed!
Polls measure *preferences* of decided voters, and *how many* undecided voters. That's it!
Elections, unlike polls, don't include undecideds. This means variables have changed!
Check out these polls from NV, 2018. How about NV-2? Poll is 16-23. Meaning ~61% undecided
Does this poll suggest that Amodei will probably win? Debatable. Does it suggest he'll probably win by about 7? No! Why? It tells us *nothing* about the 61% undecided. (He won 58-42)
Does this poll suggest that Amodei will probably win? Debatable. Does it suggest he'll probably win by about 7? No! Why? It tells us *nothing* about the 61% undecided. (He won 58-42)
According to @FiveThirtyEight's definition of polling error, a 16-23 poll is basically the same as a 43-50 one: +7.
By their definition, a poll that says 16-23 (+7), where the result is 42-58 (+16), represents a nine point polling error. Nine!
Now, why does this matter?
By their definition, a poll that says 16-23 (+7), where the result is 42-58 (+16), represents a nine point polling error. Nine!
Now, why does this matter?
Because @FiveThirtyEight and @NateSilver538 himself still say things like "if there's a polling error the same size as 2016..."
This fundamental misunderstanding of how polls work is bad enough when you're just a stats nerd. But he has a huge platform. He's spreading false info.
This fundamental misunderstanding of how polls work is bad enough when you're just a stats nerd. But he has a huge platform. He's spreading false info.
It's not as though this is a hard concept: Polls don't attempt to measure how/if undecideds will eventually vote. We all can agree on that.
Yet when you say the poll margin is supposed to measure the election margin (calling the discrepancy an error) you're ignoring undecideds!
Yet when you say the poll margin is supposed to measure the election margin (calling the discrepancy an error) you're ignoring undecideds!
It's such a common error in experimental design (and statistical analysis) that there's even a name for it: internally invalid
There's an extraneous factor (undecided voters) NOT attempted to be measured by polls which can cause some or all of the "error" that they've "measured"
There's an extraneous factor (undecided voters) NOT attempted to be measured by polls which can cause some or all of the "error" that they've "measured"
So what happens? Ppl probably smarter than me just kind of *accept* this flawed notion that poll margins attempt to predict election margi
They...don't. If I come across as angry or hostile it's just because 538 has a powerful platform to educate and have instead spread misinfo.
They...don't. If I come across as angry or hostile it's just because 538 has a powerful platform to educate and have instead spread misinfo.
As much as I may come across as an a-hole on Twitter (and sometimes I am)
I *really do* care about education, a lot. I think people can understand this stuff - at least the basics.
So when I see very smart people using 538 "poll margin" logic, what should I do? Lol
I *really do* care about education, a lot. I think people can understand this stuff - at least the basics.
So when I see very smart people using 538 "poll margin" logic, what should I do? Lol
• • •
Missing some Tweet in this thread? You can try to
force a refresh
