Writing for @wired, @pomeranian99 gets a first-of-its-kind behind-the-scenes look at Youtube's algorithm development team, in order to document the company's attempt to reduce the service's role in spreading and reinforcing conspiracy theories.
Thompson traces the origin of the crisis to the company's drive for more "engagement" that led them to tune their recommendation system to identify and propose more specialized, esoteric versions of the video you'd just watched.
2/
The idea was to lead you down a rabbit hole of ever-more-specific versions of your interests, helping you discover niches you never knew existed.
3/
This dynamic in recommendation systems has gotten a lot of attention lately, and most of it is negative, but let's pause for a moment and talk about what this means for non-conspiratorial beliefs.
4/
Say you happen upon a woodworking video, maybe due to a friend's post on social media. You watch a few of them and you find yourself interested in the subject and tuning in more often.
5/
The recommendation system presents an array of possible next-views, but tilted away from general-interest woodworking videos, instead offering you a menu of specialized woodworking styles, like Japanese woodworking.
6/
You sample one of these and find it fascinating, so you start watching more of these. The recommendation system clues you in to Japanese nail-free joinery:
7/
And from there, you discover the frankly mesmerizing "Niju-mizu-kumi-tsugi" style of joinery, and you start to seek out more. You have found this narrow, weird, self-reinforcing community.
This community could not exist without the internet and its signature power to locate and connect people with shared, widely dispersed, uncommon interests.
This power isn't just used to push conspiracies and woodworking techniques, either.
9/
It's how people who know that their gender identity doesn't correspond to the gender they were assigned at birth find each other, and acquire a vocabulary for describing their views, and foment change.
10/
It's also how people who wanted to cosplay Civil War soldiers in Charlottesville, waving tiki torches, chanting "Jews will not replace us" found each other.
11/
And that is the conundrum of the recommendation engine. Helping people find others who share their views, passions and concerns is not, in and of itself, bad. It is vital. It's the thing that made the internet delightfully weird. It's also what made the world terribly weird.
12/
Thompson takes us inside Youtube's algorithm team as they try to balance three priorities:
I. Increasing their traffic and profits
II. Helping people find others with common interest
III. Stopping conspiracies from spreading
13/
And he traces how they try, with limited success, to manage these competing goals by creating extremely fine-grained rules that define what is banned on the platform.
14/
But naturally, this just gives rise to a new kind of content: stuff that is ALMOST bad enough for blocking, but not quite. The problem is that this stuff is indistinguishable (in all but the narrowest, technical way) from banned content.
15/
So then Youtube has to create a new set of moderation guidelines: "What is so close to prohibited content that it, too, is prohibited?"
Naturally, this is creating a new kind of content: "Stuff that is not close-to-bannable, but is close-to-close-to-bannable."
16/
This dynamic should be familiar to anyone who's watched the moderation policies of Big Tech platforms evolve: what is "hate speech?" "What is 'almost-hate-speech?'" "What is almost-almost-hate-speech?'"
17/
Ultimately, this ends up creating thick binders of pseudo-law that delivers advantages to the worst people: they can study the companies' policies and figure out how to skate RIGHT UP to the cliff's edge (no matter how it is defined).
18/
And at the same time, they can goad their adversaries - the people they torment - into crossing these fractally complex lines and then nark them out, so that over time, these speech policies preferentially block good speech and leave bad speech untouched.
19/
I am increasingly convinced that the problem isn't that Youtube is unsuited to moderating the video choices of a billion users - it's that no one is suited to this challenge.
20/
Remedies that put moderation choices closer to the user - breaking up monopolies, allowing interoperable recommendation systems - solve the problem of scaling up AND covering edge cases by eliminating scale altogether, and letting the edge cases make their own calls.
21/
Image:
Mark Sargent
eof/
• • •
Missing some Tweet in this thread? You can try to
force a refresh
In other words, the world we're living in is the best possible world, and the fact that you got contact burns from collapsing on the scorching sidewalk outside of the grocery store where you couldn't afford your weekly shopping is unfortunate, but unavoidable.
3/
*Corporate Bullshit: Exposing the Lies and Half-Truths That Protect Profit, Power, and Wealth in America* is @NickHanauer, @joanwalsh and @donaldrcohen12's 2023 book on the history of corporate apologetics; it's great:
I found out about this book last fall when @ddayen reviewed it for the @TheProspect; Dayen did a great job of breaking down its thesis, and I picked it up for my newsletter.
3/
Last weekend, I was at @Defcon 32, where I had the privilege of giving a talk: "Disenshittify or die! How hackers can seize the means of computation and build a new, good internet that is hardened against our asshole bosses' insatiable horniness for enshittification."
1/
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on , my surveillance-free, ad-free, tracker-free blog:
This was a followup to last year's talk, "An Audacious Plan to Halt the Internet's Enshittification," a talk that kicked off a lot of international interest in my analysis of platform decay ("enshittification"):
Once you learn about the "collective action problem," you see it everywhere. Democrats - including elected officials - wanted Biden to step down, but no one wanted to be the first one to take a firm stand, so for months, his campaign limped on: a collective action problem.
1/
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on , my surveillance-free, ad-free, tracker-free blog:
Patent trolls use bullshit patents to shake down small businesses, demanding "license fees" that are high, but much lower than the cost of challenging the patent and getting it revoked.
3/
Every performer and creator on Patreon is about to get screwed out of 30% of their gross earnings, which will be diverted to Apple, the most valuable company on Earth. Apple contributes nothing to their work, but it can steal a third of their wages:
Enshittification starts with companies being good to end users. In this case, Apple made a quality product - Iphones - and carefully tended its App Store.
Walmart didn't just *happen*. The rise of Walmart - and Amazon, its online successor - was the result of a specific policy choice, the decision by the Reagan administration not to enforce a key antitrust law.
1/
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on , my surveillance-free, ad-free, tracker-free blog: