Thread by @cinyc9

There's a new pollster in town called SoCal Research. They're currently included in the 538 average.

But who are they and how are they actually polling?

A 🧵

1/11

According to the methodology statement from SoCal Research's MI poll, that poll "was conducted online using the Pollfish panel", and was weighted by "race, education, age, gender, CD and vote history."

2/11

How much did the poll cost? @KenJames779 asked the account running the operation, SoCal Data (nee SoCal Populist): $475.

Knowing how much it usually costs @RRHElections to poll, that seemed way too low for a quality poll. So I signed up for a Pollfish account to investigate
3/11

It turns out, yes, you can empanel a 500-respondent MI Pollfish panel for exactly $475.

However, this includes no screening questions and no meaningful registered voter screen. Yes, the RV box is checked here, but a Yes or No option isn't selected.

4/11

When the RV box is properly populated to include only RVs, the cost balloons to $750 - more than SoCal Research says they paid for the poll.

5/11

This is based on Pollfish's basic plan. Perhaps SoCal Research is on their Elite tier, where costs are lower. I don't know. But if they paid what they said they paid on the Basic tier, there appears to have been no effective RV screen - i.e. the poll is of adults.

6/11

SoCal's methodology statement says that "respondents were discarded if they were not registered to vote." But, playing around with Pollfish, a real screening question to screen out non-RVs costs ~$250, and if they empaneled more than 500, it would have cost more than $475.

7/11

As for Pollfish's methodology generally, they recruit an online panel that doesn't pay respondents cash, but instead allows respondents to unlock news content or in-app purchases. Many online panels pay-for-response, so it's not unique in that.

8/11pollfish.com/survey-target-…

This poll, however, reminds me of the crappy one-question Google Consumer Surveys some of us on the old US Election Atlas Forum conducted back in 2016. The GCS polls pulled from the same set of panelists: news site unlock questions and in-app.



9/11talkelections.org/FORUM/index.ph…

As the results on the last page show, these polls were methodologically suspect. It was GIGO & GCS eventually banned political polling.

tldr: a $475 poll could be right. But you usually get what you pay for - and the MI poll appears to be polling adults, not RVs.

10/11

Note I said appears. But if I'm right, so usually is Hanlon's Razor - SoCal might not know the mistake they appear to have made here.

Full disclosure: Note that SoCal Populist and I have some past history. They were involved in an attempt to spread fake DMs I never wrote.

11/11