Thread by @cravennewworld

Don't let #polls determine whether or not you vote. Don't assume a victory or a loss based on current polling. Why? Because they are inherently flawed, and not truly reflective of a whole group of people of any sort. (more)

2/ Polls have two major aspects: Sample and Methodology. These are the parameters that define how the poll is taken and how accurately it reflects a population.

3/ Let's start with sample, also called sample size. It tells you how many people are polled to get the results. Most of the big polls you see in the news have a sample size somewhere around a thousand. This works for the pollsters, because it's a manageable

4/ number, but for most cases, it's not a representative sampling. If you poll a thousand people to reflect a group of multiple millions of voters, that's less than .01% of voters. Yes, one-hundredth of a percent. A penny against a hundred-dollar bill.

5/ If those samples were truly random, they might be a telling trend, but they're not. Those phone lists have to be acquired somewhere, and there are a lot of folks who won't appear on them. For example, people with no land lines or

6/ people who haven't voted before. In many cases, these polls are further limited by party affiliation. For example: a thousand voters, with 400 Democrats, 350 Republicans, 250 Independents is going to show very different results than a poll with 750R 250D,

7/ or 500I 250D 250R, Then you approach race, education, age, gender, etc., and by the time you're done, that thousand people may contain, say, 200 Black voters, 150 of them over 25, 75 of them with a college degree, 37 of them female...

8/ By the time you finish slicing the numbers, it may be less than 10 people in a given segment who are being considered a representative sample. Are 10 Black women 35-49 with 6 or more years of college representative of all Black women of those ages with a PhD/MD?

9/ Sampling sizes more or less invalidate polling, imo

Now we move on to methodology.

Methodology includes some of the aspects above, but also includes such things as where the call lists for the polls came from, how they are administered, and how clear the questions are.

10/ We're probably all seen polls where the questions have no really good answer, or no really bad answer, or which are written in an unclear way. This is sometimes done by design to get the desired results. If you ask people questions with nested negatives, it becomes confusing.

11/ If a poll's questions are slanted toward a certain outcome, a respondent will have to choose the least-bad option rather than the good or accurate one. If the questions are done in those nested negatives (e.g. 'Are you not against the repeal of Measure Blah?'), it becomes a

12/ matter of having to unravel the words to get at the actual meaning. It's confusing, and that's no accident.

A truly neutral poll will be written with clarity and obvious delineation between choices. How many of them are?

13/ On to calling lists.

Polling organizations get their lists of respondents in a number of ways, but most come from your voter registration, IIRC. They will only use the information provided, and can only work within that dataset.

14/ They also tend to be done during certain hours on certain days. You won't get poll calls at 3AM, for example. This means that people who work second- or third-shift, or people who work multiple jobs, or people who work and go to college, or

15/ people who are single parents, to name just a few, will be under-represented,

They also depend on a person's willingness to participate. If you work two jobs, or have kids, or are caring for an ailing relative, or are in school, how much of your precious and limited time

16/ are you willing to give to a poll? How much time can you spare?

Consider, then, the people who *can* spare the time: people with 9-5 jobs. The retired. The sample is going to skew older and wealthier for just these reasons.

17/ Another note is that polling is dependent on land lines. Many people under 35 don't even have a land line. Others have them but use cell numbers as the primary contact number. Those people will be sampled out, largely. This means you have to rely on people who

18/ primarily use a land line, which skews polls older, more rural, or less tech-savvy.

When you consider all of this, it's hard to imagine how you can get a representative sample, even with a very well-curated phone list.

19/ The margin of error on a poll is expressed as a percentage. This reflects any potential mistakes or anomalies in the compilation of data. Usually you will see them around 2-3 percent, which is still a matter of 40-60 responses in a 1K sample. Anything above 3 percent is

20/ pretty much worthless, in a sample size this small. Margin of error applies in both directions, so it's actually twice as large as it seems: 46 percent +/- 2 percent covers the range from 44 - 48 percent. That's 4 percentage points, or 40 people in a thousand.

21/ I could probably go on about this all day, but the short summary is this:

22/ Polls are interesting, but they're not as reflective of the general populace as they're made out to be. They should not, ever, make you decide not to vote. Let's remember 2016. #EveryVoteCounts when an election can be turned by a number of votes less than a poll sample size.

23/ Vote. Get your friends, family, acquaintances, dog walker, mailman, whoever, to vote. Let's see what the majority of us choose for America.

#VoteBlueToSaveAmerica

fivethirtyeight.com/features/what-… gives a pretty clear picture of the matter.