Thread by @doctorow

The mainstream critique of Facebook is surprisingly compatible with Facebook's own narrative about its products. FB critics say that the company's machine learning and data-gathering slides disinformation past users' critical faculties, poisoning their minds.

1/

If you'd like an unrolled version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

pluralistic.net/2021/09/30/don…

2/

Meanwhile, Facebook itself tells advertisers that it can use data and machine learning to slide past users' critical faculties, convincing them to buy stuff.

3/

In other words, the mainline of Facebook critics start from the presumption that FB is a really GOOD product and that advertisers are DEFINITELY getting their money's worth when they shower billions on the company.

4/

Which is weird, because these same critics (rightfully) point out that Facebook lies all the time, about everything. It would be bizarre if the only time FB was telling the truth was when it was boasting about how valuable its ad-tech is.

5/

Facebook has a conflicted relationship with this critique. I'm sure they'd rather not be characterized as a brainwashing system that turns good people into monsters, but not when the choice is between "brainwashers" and "con-artists selling garbage to credulous ad execs."

6/

As FB investor and board member Peter Thiel puts it: "I’d rather be seen as evil than incompetent." In other words, the important word in "evil genius" is "genius," not "evil."

https://twitter.com/doctorow/status/1440312271511568393

7/

The accord of tech critics and techbros gives rise to a curious hybrid, aptly named by @mariafarrell: the Prodigal Techbro.

A prodigal techbro is a self-styled wizard of machine-learning/surveillance mind control who has see the error of his ways.

crookedtimber.org/2020/09/23/sto…

8/

This high-tech sorcerer doesn't disclaim his magical powers - rather, he pledges to use them for good, to fight the evil sorcerers who invented a mind-control ray to sell your nephew a fidget-spinner, then let Robert Mercer hijack it to turn your uncle into a Qanon racist.

9/

There's a great name for this critique, criticism that takes its subjects' claims to genius at face value: #critihype, coined by @STS_News, describing a discourse that turns critics into "the professional concern trolls of technoculture."

sts-news.medium.com/youre-doing-it…

10/

The thing is, Facebook really IS terrible - but not because it uses machine learning to brainwash boomers into iodine-guzzling Qnuts. And likewise, there really IS a problem with conspiratorial, racist, science-denying, epistemologically chaotic conspiratorialism.

11/

Addressing that problem requires that we understand the direction of the causal arrow - that we understand whether Facebook is the cause or the effect of the crisis, and what role it plays.

12/

"Facebook wizards turned boomers into orcs" is a comforting tale, in that it implies that we need merely to fix Facebook and the orcs will turn back into our cuddly grandparents and get their shots. The reality is a lot gnarlier and, sadly, less comforting.

13/

There's been a lot written about Facebook's sell-job to advertisers, but less about the concern over "disinformation." In a new, excellent longread for @harpersbazaarus, @Bernstein makes the connection between the two:

harpers.org/archive/2021/0…

14/

Fundamentally: if we question whether Facebook ads work, we should also question whether the disinformation campaigns that run amok on the platform are any more effective.

15/

Bernstein starts by reminding us of the ad industry's one indisputable claim to persuasive powers: ad salespeople are really good at convincing ad buyers that ads work.

16/

Think of department store magnate John Wanamaker's lament that "Half the money I spend on advertising is wasted; the trouble is I don’t know which half." Whoever convinced him that he was ONLY wasting half his ad spend was a true virtuoso of the con.

17/

As @timhwang documents brilliantly in his 2020 pamphlet "Subprime Attention Crisis," ad-tech is even griftier than the traditional ad industry. Ad-tech companies charge advertisers for ads that are never served, or never rendered, or never seen.

pluralistic.net/2020/10/05/flo…

18/

They rig ad auctions, fake their reach numbers, fake their conversions (they also lie to publishers about how much they've taken in for serving ads on their pages and short change them by millions).

19/

Bernstein cites Hwang's work, and says, essentially, shouldn't this apply to "disinformation?"

If ads don't work well, then maybe political ads don't work well. And if regular ads are a swamp of fraudulently inflated reach numbers, wouldn't that be true of political ads?

20/

Bernstein talks about the history of ads as a political tool, starting with Eisenhower's 1952 "Answers America" campaign, designed and executed at great expense by Madison Ave giants Ted Bates.

21/

Hannah Arendt, whom no one can accuse of being soft on the consequences of propaganda, was skeptical of this kind of enterprise: "The psychological premise of human manipulability has become one of the chief wares that are sold on the market of common and learned opinion."

22/

The ad industry ran an ambitious campaign to give scientific credibility to its products. As Jacques Ellul wrote in 1962, propagandists were engaged in "the increasing attempt to control its use, measure its results, define its effects."

23/

Appropriating the jargon of behavioral scientists let ad execs "assert audiences, like workers in a Taylorized workplace, need not be persuaded through reason, but could be trained through repetition to adopt the new consumption habits desired by the sellers." -Zoe Sherman

24/

These "scientific ads" had their own criti-hype attackers, like Vance "Hidden Persuaders" Packard, who admitted that "researchers were sometimes prone to oversell themselves—or in a sense to exploit the exploiters."

25/

Packard cites Yale's John Dollard, a scientific ad consultant, who accused his colleagues of promising advertisers "a mild form of omnipotence," which was "well received."

26/

Today's scientific persuaders aren't in a much better place than Dollard or Packard. Despite all the talk of political disinformation's reach, a 2017 study found "sharing articles from fake news domains was a rare activity" affecting <10% of users.

science.org/doi/10.1126/sc…

27/

So, how harmful is this? One study estimates "if one fake news article were about as persuasive as one TV campaign ad, the fake news in our database would have changed vote shares by an amount on the order of hundredths of a percentage point."

aeaweb.org/articles?id=10…

28/

Now, all that said, American politics certainly feel and act differently today than in years previous. The key question: "is social media creating new types of people, or simply revealing long-obscured types of people to a segment of the public unaccustomed to seeing them?"

29/

After all, American politics has always had its "paranoid style," and the American right has always had a sizable tendency towards unhinged conspiratorialism, from the John Birch Society to Goldwater Republicans.

30/

Social media may not be making more of these yahoos, but rather, making them visible to the wider world, and to each other, allowing them to make common cause and mobilize their adherents (say, to carry tiki torches through Charlottesville in Nazi cosplay).

31/

If that's true, then elite calls to "fight disinformation" are unlikely to do much, except possibly inflaming things. If "disinformation" is really people finding each other (not infecting each other) labelling their posts as "disinformation" won't change their minds.

32/

Worse, plans like the Biden admin's National Strategy for Countering Domestic Terrorism lump 1/6 insurrectionists in with anti-pipeline activists, racial justice campaigners, and animal rights groups.

33/

Whatever new powers we hand over to fight disinformation will be felt most by people without deep-pocketed backers who'll foot the bill for crack lawyers.

34/

Here's the key to Bernstein's argument: "One reason to grant Silicon Valley’s assumptions about our mechanistic persuadability is that it prevents us from thinking too hard about the role we play in taking up and believing the things we want to believe.

35/

"It turns a huge question about the nature of democracy in the digital age—what if the people believe crazy things, and now everyone knows it?—into a technocratic negotiation between tech companies, media companies, think tanks, and universities."

36/

I want to "Yes, and" that.

My 2020 book How To Destroy Surveillance Capitalism doesn't dismiss the idea that conspiratorialism is on the rise, nor that tech companies are playing a key role in that rise - but without engaging in criti-hype.

onezero.medium.com/how-to-destroy…

37/

In my book, I propose that conspiratorialism isn't a crisis of what people believe so much as how they arrive at their beliefs - it's an "epistemological crisis."

We live in a complex society plagued by high-stakes questions none of us can answer on our own.

38/

Do vaccines work? Is oxycontin addictive? Should I wear a mask? Can we fight covid by sanitizing surfaces? Will distance ed make my kind an ignoramus? Should I fly in a 737 Max?

Even if you have the background to answer one of these questions, no one can answer all of them.

39/

Instead, we have a process: neutral expert agencies use truth-seeking procedures to sort of competing claims, showing their work and recusing themselves when they have conflicts, and revising their conclusions in light of new evidence.

40/

It's pretty clear that this process is breaking down. As companies (led by the tech industry) merge with one another to form monopolies, they hijack their regulators and turn truth-seeking into an auction, where shareholder preferences trump evidence.

41/

This perversion of truth has consequences - take the FDA's willingness to accept the expensively manufactured evidence of Oxycontin's safety, a corrupt act that kickstarted the opioid epidemic, which has killed 800,000 Americans to date.

42/

If the best argument for vaccine safety and efficacy is "We used the same process and experts as pronounced judgement on Oxy" then it's not unreasonable to be skeptical - especially if you're still coping with the trauma of lost loved ones.

43/

As @annamerlan writes in her excellent Republic of Lies, conspiratorialism feeds on distrust and trauma, and we've got plenty of legitimate reasons to experience both.

memex.craphound.com/2019/09/21/rep…

44/

Tech was an early adopter of monopolistic tactics - the Apple ][+ went on sale the same year Ronald Reagan hit the campaign trail, and the industry's growth tracked perfectly with the dismantling of antitrust enforcement over the past 40 years.

45/

What's more, while tech may not persuade people, it is indisputably good at FINDING them. If you're an advertiser looking for people who recently looked at fridge reviews, tech finds them for you. If you're a boomer looking for your old high school chums, it'll do that too.

46/

Seen in that light, "online radicalization" stops looking like the result of mind control, instead showing itself to be a kind of homecoming - finding the people who share your interests, a common online experience we can all relate to.

47/

I found out about Bernstein's article from the @techdirt podcast, where he had a fascinating discussion with host @mmasnick.

techdirt.com/articles/20210…

48/

Towards the end of that discussion, they talked about FB's #ProjectAmplify, in which the company tweaked its news algorithm to uprank positive stories about Facebook, including stories its own PR department wrote.

pluralistic.net/2021/09/22/kro…

49/

Project Amplify is part of a larger, aggressive image-control effort by the company, which has included shuttering internal transparency portals, providing bad data to researchers, and suing independent auditors who tracked its promises.

50/

I'd always assumed that this truth-suppression and wanton fraud was about hiding how bad the platform's disinformation problem was.

But listening to Masnick and Bernstein, I suddenly realized there was another explanation.

51/

Maybe Facebook's aggressive suppression of accurate assessments of disinformation on its platform are driven by a desire to hide how expensive (and profitable) political advertising it depends on is pretty useless.

52/

Image:
Anthony Quintano (modified)
commons.wikimedia.org/wiki/File:Mark…

Cryteria (modified)
commons.wikimedia.org/wiki/File:HAL9…

CC BY:
creativecommons.org/licenses/by/3.…

eof/