Thread by @brianfishman

[Work Thread] A version of Facebook’s Dangerous Organizations and Individuals list was leaked today. I want to provide some context, especially about our legal obligations, and point out some inaccuracies and mischaracterizations in the coverage. 1/n

First, Facebook does not want violence organized or facilitated on its platform and the DOI list is an effort to keep highly risky groups from doing that. It’s not perfect, but that’s why it exists. 2/n

Second, the leaked list is not comprehensive. That matters b/c the list is constantly being updated as teams try to mitigate risk. Like other tech comps, we havent shared the list to limit legal risk, limit security risks, & minimize opportunities for groups to circumvent rules.

Third, the DOI policy is unusual for two reasons: actors are subject to it based on behavior off of Facebook, not just what they post on-platform, and in some cases we have a legal obligation to remove these actors. 4/n

That latter issue is really important and poorly understood. For example, FB has a legal obligation to follow U.S. law related to entities designated as foreign terrorist organizations, global terrorists and other sanctioned parties. 5/n

In our Community Standards, we’ve made it clear where these legal requirements overlap with our DOI policy. 6/n

Defining & identifying Dangerous Orgs globally is extremely difficult. There are no hard & fast definitions agreed upon by everyone. Take terrorism. Government lists reflect their political outlook and policy goals. And even government agencies define the problem differently. 7/n

Likewise, the UN hasn’t settled on a universal definition of terrorism. The same problem applies across all the other DOI sub-categories. That’s why we make our definitions public in the Community Standards and it’s why we go beyond (and don’t simply follow) government lists. 8/n

Many of the groups listed in the document are subsidiaries or media wings of larger entities. This is particularly true with well-established terrorist groups like ISIS and al-Qaeda, for which we have documented hundreds of individual entities. 9/n

That matters because in a superficial analysis this structure dramatically skews the overall number of entities from a particular region. 10/n

It’s important that FB document each ISIS Wilayat to facilitate enforcement, but counting each separately to support the argument that the overall list is biased is misleading. 11/n

That kind of analysis generates heat, not light, and is an inaccurate and misleading reflection of the groups on our ‘Tier one’ list. 12/n

In general, this sort of superficial analysis does not reflect the reality of our legal obligations or the fact that hundreds of the listed entities are derivative of our designations of uncontroversial orgs like ISIS and al-Qaeda. 13/n

The data analysis matters because The Intercept’s story uses those misleading figures to suggest that FB doesn’t take white supremacist groups seriously. That is wrong. 14/n

More than 250 white supremacist entities are designated under our most restrictive policy as hate groups. Tier 1. That means no ‘praise, support and representation’. 15/n

In 2020 we expanded our DOI policy to include ‘Militarized Social Movements’, including militias. As far as we know, this is also the most comprehensive list of its kind and reflects the legitimate concern we had regarding potential political violence in 2020. 16/n

But if a would-be ‘MSM’ meets the bar to be designated under our most restrictive policies they are. For example, the Proud Boys were listed as a hate org in 2018. 17/n

I don’t want to suggest that FB’s Dangerous Orgs list, or its enforcement, is perfect. It isn’t. We don’t get to every organization as quickly as we’d like and the policy, as we’ve long stated, is blunt. Enforcement is not perfect, in part because groups are adversarial. 18/n

Sometimes discussion of content moderation gets framed as if there are permanent, perfect solutions. That’s not right. Every day it is a matter of adversarial adaptation. We can’t get rid of these groups completely, but we can make it harder for them to operate. 19/n

Finally, I want to reflect on something here that I wrote to the internal teams yesterday. 20/n

I don’t condone this leak. In the aggregate it makes everything harder. There will be criticism. But we’ll use it as an opportunity to get better. 21/n

That means hard questions internally and more focused discussions with stakeholders, who often have valuable points of view. For example, the nuance we’ve added to our “Violent Non-State Actors” policy is a result of those past discussions. 22/n

In this process, we’re going to learn about gaps in designation & enforcement, & places where the policy could, & perhaps should, have more nuance. This is positive & we will use any learnings to improve FB as a platform & support more productive communities online and off. /end