Thread by @samidh

While I had no involvement whatsoever in @JeffHorwitz's very thorough reporting in the WSJ on FB's x-check system, I was quoted in the article based on a leaked internal post, so I am compelled to give a more full perspective.

https://twitter.com/JeffHorwitz/status/1437422025258188801

First, to state the obvious, automated moderation systems inevitably make lots of mistakes because human language is nuanced & complex. In theory, a confirmatory round of review is prudent because it is an awful experience to have your post taken down without cause.

But how you execute that second round of review is critically important! Figuring out who is eligible, how you staff, etc. makes all the difference between responsible enforcement and de-facto exemptions from the platform's policies.

Though it is easy to say there aren't two systems of justice on FB in terms of the policies themselves, having two different *processes* for justice is equally problematic. As they say, justice delayed is often justice denied. And process *is* law.

So how do you reconcile this essentially impossible accuracy vs. latency vs. resourcing tradeoff? There's no ideal solution, but here are some things that could help: 1/ list transparency, 2/ adequate staffing, and 3/ better user messaging & appeals.

On 1/ Write a publicly defensible and transparent standard for who is eligible for x-check-style reviews. Then make both the standard and the lists visible to all both to ensure those standards are being met and to evolve the standards over time.

On 2/ Adequately staff the human moderation queue to guarantee a maximum latency (SLA) for reviews. If a reasonable SLA can't be achieved, make the eligibility criteria more stringent. While pending review, reduce the distribution of potentially violating posts to minimize harm.

On 3/ Failing 1 & 2, fallback on automation and pair it with more informative messaging to users as well as an expedited and robust appeals process. People should know their posts were taken down due to an algorithm and get an equitable chance for a second look.

Steps like these would not be free of controversy. Critics will assume worst intent and assail the company for imagined favoritism and bias. This is where uninformed criticism can actually backfire and prevent platforms from doing the brave thing for society.

Fundamentally, the trust & safety solution space is far broader when platforms prioritize reduction of user harm over short-term reputational concerns. When integrity teams get frustrated with company leaders, it is usually due to a misalignment on which priority matters more.

Finally, I do wish the WSJ article had better highlighted the progress FB has made already, including on the issues flagged above. There is more to do and I hope that this moment of broader reflection is an opportunity for the teams doing this hard work to get even more done.