100 opinions on political data, from a few years working in it and a few years working outside of it looking in.
Starting with:
1. From an analyst perspective, election data is trash.
Starting with:
1. From an analyst perspective, election data is trash.
You get real data on what you care about like 1-2 times a year, it’s only IF someone voted (not for whom), and so you have to do a lot of modeling off of other models rather than using actual data.
2. People would probably be horrified to learn how much data basically anyone can buy off of a Secretary of State’s office. That stuff forms the backbone of voter data files (name, address, etc)
3. Where it gets hard/interesting, depending on viewpoint, is joining a bunch of data together and having it attach to the same person. It is many peoples’ jobs to figure out how to efficiently determine which Jane Smith from New Hampshire you are.
4. This is also why I always laugh when people tell me how cool their analytical approaches are. Like, maybe certain kinds of machine learning are cool, but so much of it is down to how good your data is and how much of it you have.
5. Incidentally, this is why Google and Facebook are zillion dollar businesses: they can connect the dots better than anyone. Turns out the best way to make money from that position is sell highly targeted ads, but if anything is better in the future, they can do that too.
6. For the rest of the orgs, corporate and political, it’s a buy-or-build world. Most people opt to buy, which means that most political orgs on the left use one of three or so mega-files, and then they just build models on top of them.
7. Because consultants gotta consult, those models tend to be proprietary, except for the ones built for the party. This is generally pretty inefficient for political orgs and good for consultants.
8. All of that is pretty meh, though, compared to how dogshit political polling data is. It was bad in 2014 and I can only imagine how much worse it is now.
9. The quickest way I can convey how shit polling data is is the following:
1. Sampling as a science (polls) rely on a key assumption: the sample’s average is pretty close to the population’s average.
2. How you get that sample is uncontrollably biased.
3. Lol polls basically
1. Sampling as a science (polls) rely on a key assumption: the sample’s average is pretty close to the population’s average.
2. How you get that sample is uncontrollably biased.
3. Lol polls basically
10. This is how you get shit like 2016, where it turns out the industry that most people rely on for accurate info was using super high variance data (calling people and hoping they pick up). Anyone still pick up their phone from random numbers? Me either.
11. This means that the people who DO are actually the weird ones. Oh, and for legal reasons, you can’t call random cell phones using various methods that some polls use, so cell phone polls are more expensive, so fewer places use them. Landlines forever!
12. But lots of people don’t and never have had a landline, yours truly included. So I’m pretty hard to poll. Enter online polls, including panels. Panels are a good way to get unknown-bias, low-variance data (look up bias-variance tradeoff on Wikipedia if you want more on this)
13. Anyway, no idea what polling looks like nowadays, but it’s probably pretty methodologically fraught, which makes it hard to compare results. And anyways, you don’t know who voted for whom in the past unless you have literally talked to them on a poll, but good luck there.
14. Oh, also, people lie to pollsters. About lots of things, but if people voted like they said they were going to in phone polls, we’d have a much healthier republic.
15. People use polls to mean lots of things, but the level of sophistication in reading polls is F minus. People really don’t have a good way to understand probabilities that aren’t 0, 1, or .5 - turns out that 87% to win means that you can lose three coin flips in a row and lose
16. Alternatively, imagine rolling dice at a casino. The odds of rolling a 7 are 16.6%, so ballpark Trump’s odds a few days before the 16 election, according to some models. But elections only get run once per cycle, so I guess the USA...crapped out.
17. From a data science sophistication perspective, I think politics is probably about on par with corporate America. Where they’re weaker is access to decent data, but you really need to have a consumer product in order to collect it, so bit of a catch-22.
18. Very hot takey, but this is why I always thought there was opportunity space in getting into the data-harvesting app space for political orgs. Not like people use your election year apps anytime outside of election season...
19. People in general don’t understand the difference between analytics polling (who are you going to vote for) and messaging polling (who ya got, ok they’re a socialist, now who ya got). Former wants to be pristine, latter wants to measure impact of message.
20. Because people have too much Reynolds wrap, though, they think “omg, bad person on phone is trying to persuade me that X is a socialist! I hate push polls!” Close, but they just want to know how to make their TV ads. Closest thing to A/B testing messages they can get.
21. Speaking of A/B tests, I feel for the analysts who run tests, get results, and then have to hope that those comms channels are still effective the next time they have to use them YEARS later.
Ok, putting a pin in this. Reminder for self in the morning: political binary variables, kalla/broockman, IE firewalls
Love sleeping in.
22. American political discourse is horribly stunted by the political reality of two parties, and so is analytics, at least as of 5 years ago. Turns out fitting a model on “are you an R or a D?” blacks out a lot of gray that could be valuable.
22. American political discourse is horribly stunted by the political reality of two parties, and so is analytics, at least as of 5 years ago. Turns out fitting a model on “are you an R or a D?” blacks out a lot of gray that could be valuable.
23. Partisanship models (as of 2014, on the left) were logits - 0 is R, 1 is D. Smart readers might think “huh, how do you tell the diff between Bernie and Hillary dems?” Answer: mostly with issue models. But core election predictions used those logits.
24. So how could analytics not see Trump Democrats coming? One answer: only sparse data available to predict which Ds are further right on immigration, general racist sentiments, homophobia, etc — sorry, sorry, “economic anxiety” — and it was descriptive (what do you do with it?)
25. This leads to my all time fav polisci dissertation ever, which everyone should read in full: politicians are centrists, voters are extremists on issues, and “moderates” are mostly a myth, since a lot of people don’t have a political ideology that neatly confirms to D/R axis.
26. Broockman is probably the best poli sci thinker in my generation: digitalassets.lib.berkeley.edu/etd/ucb/text/B…
27. So when you combine “voters are issue extremists” and “micro targeted ads are highly profitable and relatively newly possible”, you get to run digital campaigns with 1000 messages curated to each segment, targeted perfectly.
28. This is mostly bad, because in advertising, you used to have to own your message to many groups. You couldn’t play both sides of a fence. Now you can, because of how ad audiences work. “Divisive” messages can now just go to people who Like Stormfront on FB, for instance.
29. It also gets into why social media is good/bad/powerful: it lets communities form identity groups around their beliefs. Maybe this is good because you can find fellow DemSocs where before it wasn’t worth wading through a morass of lefty groups, but also Nazis can organize.
30. Another spicy take: social censure is mostly good and social media removes the power of it. Censure enforces norms. Absent that, people just go deeper into their filter bubbles and buy into those norms as The Good Norms. But every group has different ones, so: polarization!
31. Imagine the halcyon days of not that long ago, when you were stuck with your IRL friends and had to conform to behavior that would get them to like you. Fuck that! Now you can do whatever you want and there’s a community out there just for you.
32. This is why FB’s mission statement is bad. The world should not be fully connected. People have to live in the world they inhabit, and making it easier to escape that community’s norms is double-edged - you provide escape for persecuted outgroups, but also for fascists.
33. The endpoint of political targeting is likely some amount of coopting by the owners of geolocation, financial transaction, and communication data (ie FB and Google). Dunno how exactly it’ll happen, whether party people get into corps and make calls or what, but someday.
34. Speaking of using external resources to magnify messaging: the party/independent expenditure firewall is fucked. This is largely because the FEC is toothless and paralyzed.
35. Back in 14, we encountered several examples of Rs just blatantly passing info from IEs to campaigns via twitter accounts that got locked as soon as they got scrutinized by media. It’s a gigantic fucking joke. Can only imagine how much worse it is now.
36. This gets at another area where politics is fucked: it has been largely governed by norms (ie “bipartisanship, or effort toward it, is good and worthy”) and it turns out norms aren’t rules!
37. And the media loves norms and enforcing them, but their framework is deeply ahistorical because no one wants to sit through a history lesson to learn about who’s to blame for the FEC being fucked. So it’s easy to push bullshit and both-sidesing doesn’t offend anyone, so.
38. A lot of what’s troubling nowadays isn’t that people are lying more, but rather than media business models are driven by engagement (clicks -> ads) rather than prestige via accuracy. If you offend conservatives by pointing out that their pols lie a lot, they’ll stop clicking!
39. This creates a weird split: media loves to see themselves as truth tellers, but they mostly peddle narratives that try to balance between what they see as true + what will maintain a big audience to whom they can serve ads. Hard to defend.
40. This landscape is obviously super exploitable, but only one side is really interested in/good at exploiting it. Not sure why, but the Luntz/Rove/Bannon tradition of message crafting is almost solely a right wing thing in the US. Left is dogshit at telling a story.
41. This is why the few actual left wing pols that get elected (AOC, etc) feel like breaths of fresh air: they tell a coherent story about what’s happened, what’s going on, and what to do about it. Centrists have no good story to tell, so it’s all platitudes and nostalgia.
42. Ultimately, though, the story being told is “the world is collapsing and you should rely on us to defend you against outgroups” vs a bunch of different center and left narratives.
43. The only left narratives that stand a chance against that story are “yes, the world is collapsing, but here’s a better plan”, because it turns out people are fine with genocide if it’s to protect them and theirs. The liberal values story we tell is deeply ahistorical.
44. That doesn’t mean we can’t live up to better, only that we haven’t in the past, much as we like to say we have.
45. Straying a bit from data here. Jumping tracks here: I think that most people have deeply inconsistent views on data and privacy. We have largely been sold on a more convenient world where we don’t have privacy of any kind.
46. I participate in this too! I have given the government my fingerprints so I don’t wait in airport lines, I let major ad networks read my texts and emails so I can centralize my conversations and digital social graphs. It’s easy to use and hard to leave.
47. The inexorability of these systems moving toward authoritarian government practices, if not ideologies, is hard to fight. Smarter people than me are figuring out how to do it. It likely involves dismantling the whole ads/engagement business model that runs the thing.
48. Relatedly: I don’t know how we’d ever go back to a more “professional”/“dignified” office of the president after this. We live in an attention economy. Policy details don’t get clicks. Wrestling grudges and drama does. POTUS is a master of the latter form.
49. I think probably the biggest accelerant to the conservative movement in the US has been Fox priming people for outrage against outgroups. Nonstop world-ending drama has to change you over years. POTUS just told them what they’d been primed to hear.
50. Dunno how to deprogram those folks, but social media furthers balkanization there. Also: young people moving to cities to escape their dying conservative hometowns and not looking back.
51. Hard to give boomer conservatives any incentive to change when they see a fundamental uprooting of “their” way of life playing out in their children’s lives, the nation’s values, and growing physical social isolation.
52. This also ties into how rough losing the union fight has been. Theda Skocpol wrote some great books about the tea party and Koch groups that remade the nature of physical social networks around conservative politics, replacing unions.
53. Looking back on the 90s/00s, losing the fight for what communities organized around (from “being working class” to “culture war stuff” - guns, politicized religion, etc) was a big fuckup on the left. We’re paying for that now and will be for decades.
54. This plays into the growing city/rural divide. Young liberal minded people move to places of opportunity and shared values, mostly on coasts. I know folks from so many Midwestern places in SF. It’s where jobs and opportunity is.
55. I have yet to meet a 20/30-something who isn’t at or to the left of their parents here. Easy to see how to lose EC and win pop vote on the left - no one I know is moving to Idaho any time soon.
56. I think the opportunity for refighting the union battle is passed, though. So many manufacturing jobs are overseas now. Almost like someone saw this coming and encouraged international unions rather than having them serve as nationalist worker protection...
57. At this point, the switching costs of moving operations to other countries is super low. No one is gonna “bring jobs back” at scale - easier to build a middle class in China/India than prop ours up. So in that sense we’re pretty fucked.
58. So where is all of that money that was previously going to middle class Americans? Well, turns out right after WW2 was a great time to be in US since Europe and Asia’s wealth was mostly destroyed. Since then it’s gotten worse for us bc only one direction for things to go.
59. But also we have too many billionaires and corps have too many money for any kind of sustained livable lifestyle in the US, guillotines in the 2020s let’s fuckin go
60. But also it turns out that billionaires won’t let go of their power and will merge with an authoritarian state long before willingly giving that up (die internationale starts playing)
61. Okay off history and Marxism before it gets me canceled at work, we are a public company after all.
Back to political data: if you imagine an X-Y axis of partisanship and turnout and accept that there’s a group of people you want to target to get them to vote...
Back to political data: if you imagine an X-Y axis of partisanship and turnout and accept that there’s a group of people you want to target to get them to vote...
61b. ...your logic also dictates that there’s a group on the opposite side of that space that you should target to get them to NOT vote.
62. This is generally not something that people think is “nice” or “good” but also it’s pretty fucking SOP in politics.
63. That is: it is 101 level political field work that the right would want to suppress majority-Dem groups, either in sentiment or capability. Same with Dems and old white folks.
64. Again, as with so many things, only the right actually does this well. All the voter ID shit is fire at suppressing black and young votes, and Dems suck at fighting that and also at doing the same to the R base.
65. I get why - it’s mean and antisocial - but it speaks to a disconnect about what’s at stake. Rs get that politics is a fight over who gets to choose, writ broadly. Ds see it as a gentleman’s game of persuasion. Parasol to a gun fight level shit.
66. There are some operators that do some of that work on the left, but way too few and way less organized. It’s honestly embarrassing when you look at the stakes.
67. Also, it’s not like the left doesn’t pay the same fucking social penalties as the right on this! Everything will get framed as both-sides, so why not just auto-register people in college and make it hard for old folks to renew ID? What the fuck is going to happen to you?
68. Beyond that, there’s some basic, basic pro-social equity that helps Dems that isn’t operational in all blue states: Election Day voter reg, auto vote by mail, making it easier to vote where you go to college, etc
69. But yeah, Dems clutch their pearls way too much for no value. At some point you have to get in the mud and fight the people in front of you rather than leaning into nostalgia.
70. Switching tracks again: I think people just do not understand/like political parties as a system.
71. You can see this in opinions about primaries (open v closed), superdelegates, etc. Parties’ ROLE is to be unrepresentative power structures that enforce values.
72. Bad news for leftists, though: a lot of what the D party needs to do is persuade some leftists to give support while also making very few concessions, because the D party’s leftist wing has never really been the powerful one.
73. This is part of why the DSA came to be: a party with different values to serve a meaningfully different population of voters. Eventually it needs to clothe itself in Dem garments to get on ballots, but it’s a different party for sure.
74. The point of parties, to return to the point, is to control and enforce power. In exchange for fealty, you get resources and support. If a party can’t command those, it’s toothless.
75. This is why Nancy Pelosi is going to run the party as long as she wants. Look at her fundraising numbers and look at everyone else’s: she RUNS the show. People NEED HER. That’s power. Ideology is subservient in party politics.
76. It’s why the successful leftists are all good fundraisers: if someone can cut off your resources, they control your power. Materialist analysis remains undefeated, etc
77. This is part of why I don’t love Obama’s legacy as it relates to the D party: when faced with an opportunity to build up the D party, he gave his list to Organizing for America. Party lost out on years of organizing work and list building. Got it eventually, but still.
78. So in the end, a hugely valuable asset for political candidates is...email lists. If you can command the attention and loyalty and funds of people, you are in hock to THEM, not whoever gives you a disproportionate amount of your money.
79. This is also a crux of left politics: if you actually really want a bunch of pols who aren’t controlled by the D party, the single best thing you can do is give them money, repeatedly.
80. Otherwise, they can’t do shit, or they’ll get money from elsewhere and their ideology will be limited by their funders. No surprise that “billionaires are good actually” pols have it easier: natural fundraising constituency.
81. But yeah, parties exist to build coalitions and moderate their views. If you want a more extreme position, you’ll need to find a way to hijack resources/power or build a different coalition.
82. From this perspective, the answer to “does the dem party want Bernie to win” is obvious: he’s not in the party and doesn’t kowtow to it. He’s there to take it over. Of course they don’t want him. They want his people at as low cost as they can pay for them.
83. But no, parties want candidates that reinforce, rather than combat, their structural power. So why, then, did the R party get taken over by POTUS so thoroughly?
84. If you, like Pepperidge Farms, remember 2012, there was a postmortem that the RNC wrote on immigration that boiled down to “we gotta stop being racist af”. Turns out that was exactly wrong, and parties want to win, and POTUS didn’t challenge any of their actual ideology.
85. So in very glib and simple terms: GOP wanted to win, primaries are tests of which candidate & policy platform best accrues power, and DJT won w/ right wing immigration views. So they brought him in and he delivered in SPADES for them.
86. What has the RNC actually given up with DJT? They got their tax cuts, judges, justices, etc because he isn’t a policy wonk. He gets to do him and they can deal with kleptocracy and shitty foreign policy.
87. By contrast, I think Bernie would uproot a lot of what the modern D party is built on: taking money from social liberal m/billionaires and placating poor and subaltern folks.
88. I think Warren does some of that but isn’t fundamentally declaring war on party funders, plus she’s playing ball within the party, so not an existential threat to the D party.
89. It’s the difference between policy differences and structural differences. Both are leftists, but underneath their similar Whats are VERY different Hows.
90. It’s telling that I was asked for 100 tweets on political data and field shit and I’ve talked about social media, ads, WW2, unions, data structures, party structures, etc. Everything is political if you look at it hard enough.
91. To tie back to why I have most of my following: I think the nerd space is under-served by overt leftist discourse.
92. There’s so much space for leftists to talk about “hate microtransactions? Actually, you hate capitalism!” on let’s play videos and streams.
93. Like, if PDP can “”””””joke”””””” about Nazi shit on his stream, there can be a leftist counterbalance of people talking about structural power, capital, labor, etc
94. But instead we have a right that jerks off people who love to feel smart/better and not think too hard about it and there is FERTILE ground for that in nerd communities. We fucking love being told that we’re smart and better than others because of it.
95. And the way that conservatism creeps in is to countersignal against leftist “tells” and insisting that their interactions aren’t political and can’t we stick to not politics? I just came here to escape politics. Lol you can never escape politics, get comfortable.
96. Also, getting back to social norms, and going back to some shit that got me canceled in 2014: the way you enforce norms is to push out people who break them. Many nerd spaces have norms around not canceling people for non-nerd transgressions.
97. That means that it’s unforgivable to cheat at Magic (lifetime bans!), but if you sexually harassed someone after the tournament? Well, people change, I know that guy he’s a good person just misunderstood, struggling with alcohol, etc.
98. What makes Magic kind of weird is that it’s still somewhat a physical-space hobby, so norms enforcement is still possible without too much fragmentation. There’s just a lot of fighting left over what norms should get enforced.
99. This is part of what companies do when they insist on Staying Out of Politics: they signal to consumers that they don’t have community values beyond Buy Our Stuff. The rest is signaling and paying the lowest price for the most audience.
100. It’s really hard to come up with 100 different thoughts about something and I cheated the format by having multiple tweets comprising a topic. Highly recommend if you want to stretch yourself.
