Thread by @doctorow

#ContentModeration is fundamentally about making social media work better, but there are two other considerations that determine how social media *fails*: #EndToEnd (#E2E), and #FreedomOfExit. 1/

These are much neglected, and that's a pity, because how a system fails is every bit as important as how it works. 2/

If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

pluralistic.net/2022/12/19/bet… 3/

Of course, commercial social media sites don't want to be *good*, they want to be *profitable*. The unique dynamics of social media allow the companies to uncouple quality from profit, and more's the pity. 4/

Social media grows thanks to #NetworkEffects - you join Twitter to hang out with the people who are there, and then other people join to hang out with you. The more users Twitter accumulates, the more users it *can* accumulate. 5/

But social media sites stay big thanks to high #SwitchingCosts: the more you have to give up to leave a social media site, the harder it is to go:

eff.org/deeplinks/2021… 6/

Nature bequeaths some in-built switching costs on social media, primarily the #CoordinationProblem of reaching consensus on where you and the people in your community should go next. The more friends you share a social media platform with, the higher these costs are. 7/

If you've ever tried to get ten friends to agree on where to go for dinner, you know how this works. Now imagine trying to get *all* your friends to agree on where to go for dinner, for the rest of their lives!

But these costs aren't insurmountable. 8/

Network effects, after all, are a double-edged sword. 9/

Some users are above-average draws for others, and if a critical mass of these important nodes in the network map depart for a new service - like, say, #Mastodon - that service becomes the presumptive successor to the existing giants. 10/

When that happens - when Mastodon becomes "the place we'll all go when Twitter finally becomes unbearable" - the downsides of network effects kick in and the double-edged sword begins to carve away at a service's user-base. 11/

It's one thing to argue about which restaurant we should go to tonight, it's another to ask whether we should join our friends at the new restaurant where they're already eating. 12/

Social media sites who want to keep their users' business walk a fine line: they can simply treat those users well, showing them the things they ask to see, not spying on them, paying to police their service to reduce harassment, etc. But these are costly choices: if you… 13/

…show users the things they *ask* to see, you can't charge businesses to show them things they *don't* want to see. If you don't spy on users, you can't sell targeting services to people who want to force them to look at things they're uninterested in. 14/

Every moderator you pay to reduce harassment draws a salary at the expense of your shareholders, and every catastrophe that moderator prevents is a catastrophe you can't turn into monetizable attention as gawking users flock to it. 15/

So social media sites are always trying to optimize their mistreatment of users, mistreating them (and thus profiting from them) right up to the point where they are ready to switch, but without actually pushing them over the edge. 16/

One way to keep dissatisfied users from leaving is by extracting a penalty from them for their disloyalty. 17/

You can lock in their data, their social relationships, or, if they're "creators" (and disproportionately likely to be key network nodes whose defection to a rival triggers mass departures from their fans), you can take their audiences hostage. 18/

The dominant social media firms all practice a low-grade, tacit form of hostage-taking. Facebook downranks content that links to other sites on the internet. Instagram prohibits links in posts, limiting creators to "#LinksInBio." Tiktok doesn't even allow links. 19/

All of this serves as a brake on high-follower users who seek to migrate their audiences to better platforms.

But these strategies are unstable. 20/

When a platform becomes worse for users (say, because it mandates nonconsensual surveillance and ramps up advertising), they may actively seek out other places on which to follow each other, and the creators they enjoy. 21/

When a rival platform emerges as the presumptive successor to an incumbent, users no longer face the friction of knowing which rival they should resettle to. 22/

When platforms' #enshittification strategies overshoot this way, users flee in droves, and then it's time for the desperate platform managers to abandon the pretense of providing a #PublicSquare. 23/

Yesterday, #ElonMusk's #Twitter rolled out a policy prohibiting users from posting links to rival platforms:

web.archive.org/web/2022121817…

This policy was explicitly aimed at preventing users from telling each other where they could be found after they leave Twitter: 24/

https://web.archive.org/web/20221219015355/https://twitter.com/TwitterSupport/status/1604531261791522817

This, in turn, was a response to many users posting regular messages explaining why they were leaving Twitter and how they could be found on other platforms. 25/

In particular, Twitter management was concerned with departures by high-follower users like @TaylorLorenz, who was *retroactively* punished for violating the policy, though it didn't exist when she violated it:

deadline.com/2022/12/washin… 26/

As Elon Musk wrote last spring: "The acid test for two competing socioeconomic systems is which side needs to build a wall to keep people from escaping? That's the bad one!"

https://twitter.com/elonmusk/status/1533616384747442176

This isn't particularly insightful. 27/

It's obvious that any system that requires high walls and punishments to stay in business isn't serving its users, whose presence is attributable to coercion, not fulfillment. Of course, the people who operate these systems have all manner of rationalizations for them. 28/

The Berlin Wall, we were told, wasn't there to keep East Germans *in* - rather, it was there to keep the teeming hordes clamoring to live in the workers' paradise *out*. 29/

In the same way, platforms will claim that they're not blocking #outlinks or #sideloading because they want to prevent users from defecting to a competitor, but rather, to protect those users from external threats. 30/

This rationalization quickly wears thin, and then new ones step in. For example, you might claim that telling your friends that you're leaving and asking them to meet you elsewhere is like "giv[ing] a talk for a corporation [and] promot[ing] other corporations": 31/

https://mobile.twitter.com/mayemusk/status/1604550452447690752

Or you might claim that it's like "running Wendy's ads [on] McDonalds property," rather than turning to your friends and saying, "The food at McDonalds sucks, let's go eat at Wendy's instead":

https://twitter.com/doctorow/status/1604559316237037568

32/

The truth is that any service that won't let you leave isn't in the business of serving you, it's in the business of harming you. 33/

The only reason to build a wall around your service - to impose *any* switching costs on users- is so that you can fuck them over without risking their departure. 34/

The platforms want to be #Anatevka, and we the villagers of #FiddlerOnTheRoof, stuck plodding the muddy, Cossack-haunted roads by the threat of losing all our friends if we try to leave:

doctorow.medium.com/how-to-leave-d…

That's where freedom of exit comes in. 35/

The public should have the right to leave, and companies should not be permitted to make that departure burdensome. Any burdens we permit companies to impose is an invitation to abuse of their users. 36/

This is why governments are handing down new #interoperability mandates: the EU's #DigitalMarketsAct forces the largest companies to offer APIs so that smaller rivals can plug into them and let users #walkaway from Big Tech into new kinds of platforms - small businesses,… 37/

…co-ops, nonprofits, hobby sites - that treat them better. These small players are overwhelmingly part of the #fediverse: the #federated social media sites that allow users to connect to one another irrespective of which server or service they use. 38/

The creators of these platforms have pledged themselves to freedom of exit. 39/

Mastodon ships with a "Move Followers" and "Move Following" feature that lets you quit one server and set up shop on another, without losing any of the accounts you follow or the accounts that follow *you*:

codingitwrong.com/2022/10/10/mig… 40/

This feature is as yet obscure, because the exodus to Mastodon is still young. Users who flocked to servers without knowing much about their managers have, by and large, not yet run into problems with the site operators. 41/

The early trickle of horror stories about petty authoritarianism from Mastodon sysops conspicuously fail to mention that if the management of a particular instance turns tyrant, you can click two links, export your whole social graph, sign up for a rival, click two more… 42/

…links and be back at it.

This feature *will* become more prominent, because there is nothing about running a Mastodon server that means that you are *good* at running a Mastodon server. 43/

Elon Musk isn't an evil genius - he's an ordinary mediocrity who lucked into a lot of power and very little accountability. 44/

Some Mastodon operators will have Musk-like tendencies that they will unleash on their users, and the difference will be that those users can click two links and move elsewhere. Bye-eee! 45/

Freedom of exit isn't just a matter of the human right of movement, it's also a labor issue. Online creators constitute a serious draw for social media services. 46/

All things being equal, these services would rather coerce creators' participation - by holding their audiences hostage - than persuade creators to remain by offering them an honest chance to ply their trade. 47/

Platforms have a variety of strategies for chaining creators to their services: in addition to making it harder for creators to coordinate with their audiences in a mass departure, platforms can use DRM, as #Audible does, to prevent creators' customers from moving the media… 48/

…they purchase to a rival's app or player.

Then there's #FreedomOfReach: platforms routinely and deceptively conflate *recommending* a creator's work with *showing that creator's work to the people who explicitly asked to see it*.

pluralistic.net/2022/12/10/e2e… 49/

When you follow or subscribe to a feed, that is not a "signal" to be mixed into the recommendation system. It's an *order*: "Show me this." Not "Show me things like this."

Show.

Me.

This.

But there's no money in showing people the things they tell you they want to see. 50/

If Amazon showed shoppers the products they searched for, they couldn't earn $31b/year on an "ad business" that fills the first six screens of results with rival products who've paid to be displayed over the product you're seeking:

pluralistic.net/2022/11/28/ens… 51/

If Spotify played you the albums you searched for, it couldn't redirect you to playlists artists have to shell out #payola to be included on:

pluralistic.net/2022/09/12/str… 52/

And if you only see what you ask for, then product managers whose #KPI is whether they entice you to "discover" something else won't get a bonus every time you fatfinger a part of your screen that navigates you away from the thing you specifically requested: 53/

doctorow.medium.com/the-fatfinger-…

Musk, meanwhile, has announced that you won't see messages from the people you follow unless they pay for Twitter Blue:

wired.com/story/what-is-… 54/

And also that you will be nonconsensually opted into seeing more "recommended" content from people you don't follow (but who can be extorted out of payola for the privilege):

socialmediatoday.com/news/Twitter-E…

Musk sees Twitter as a publisher, not a social media site: 55/

https://twitter.com/elonmusk/status/1604588904828600320

Which is why he's so indifferent to the collateral damage from this payola/hostage scam. 56/

Yes, Twitter is a place where famous and semi-famous people talk to their audiences, but it is primarily a place where those audiences talk to each other - that is, a #PublicSquare. 57/

This is the #Facebook death-spiral: charging to people to follow to reach you, and burying the things they say in a torrent of payola-funded spam. 58/

It's the vision of someone who thinks of other people as things to use - to pump up your share price or market your goods to - not worthy of consideration.

As @robandterry's #GrannyWeatherwax put it: "Sin is when you treat people like things. Including yourself. 59/

That's what sin is."

Mastodon isn't perfect, but its flaws are neither fatal nor permanent. 60/

The idea that centralized media is "easier" surely reflects the hundreds of billions of dollars that have been pumped into refining social media #RoachMotels ("users check in, but they don't check out"). 61/

Until a comparable sum has been spent refining decentralized, federated services, any claims about the impossibility of making the fediverse work for mass audiences should be treated as unfalsifiable, motivated reasoning. 62/

Meanwhile, Mastodon has gotten two things right that no other social media giant has even seriously *attempted*:

I. If you follow someone on Mastodon, you'll see everything they post; and 63/

II. If you leave a Mastodon server, you can take both your followers and the people you follow with you.

The most common criticism of Mastodon is that you must rely on individual moderators who may be underresourced, incompetent on malicious. 64/

This is indeed a serious problem, but it isn't the *same* serious problem that Twitter has. When Twitter is incompetent, malicious, or underresourced, your departure comes at a dear price. 65/

On Mastodon, your choice is: tolerate bad moderation, or click two links and move somewhere else.

On Twitter, your choice is: tolerate moderation, or lose contact with all the people you care about and all the people who care about you. 66/

The interoperability mandates in the Digital Markets Act (and in the US #ACCESSAct, which seems unlikely to get a vote in this session of Congress) only force the largest platforms to open up, but Mastodon shows us the utility of interop for smaller services, too. 67/

There are lots of domains in which "dominance" shouldn't be the sole criteria for whether you are expected to treat your customers fairly. 68/

A doctor with a small practice who leaks all ten patients' data harms those patients as surely as a hospital system with a million patients would have. 69/

A small-time wedding photographer who refuses to turn over your pictures unless you pay a surprise bill is every bit as harmful to you as a giant chain that has the same practice. 70/

As we move into the realm of smalltime, community-oriented social media servers, we should be looking to avoid the pitfalls of the social media bubble that's bursting around us. 71/

No matter what the size of the service, let's ensure that it lets us leave, and respects the #EndToEnd principle, that any two people who want to talk to each other should be allowed to do so, without interference from the people who operate their communications… 72/

…infrastructure. 73/

Image:
Cryteria (modified)
commons.wikimedia.org/wiki/File:HAL9…

CC BY 3.0
creativecommons.org/licenses/by/3.…

Heisenberg Media (modified)
commons.wikimedia.org/wiki/File:Elon…

CC BY 2.0
creativecommons.org/licenses/by/2.… 74/