It doesn't have the same radicalization backdrop, but this is TikTok's Christchurch. Both events faced troll reuploads that defeated algorithmic gating.
In response to Christchurch, YouTube's moderation infrastructure crumbled, and the company was forced to break search site-wide for days. All western platforms went scrambling; it was a different threat than the ISIL coalition had prepared them for.
The WSJ recently reported that Zhang Yiming took that very event as a foundational lesson for ByteDance's moderation efforts. While his infrastructure has meant geopolitical censorship, it is clear that TikTok is struggling today to meet this expectation. wsj.com/articles/entre… 
TikTok has had previous tests too. One of the core reasons ByteDance's investment in India grew was a result of the horrific FYP hellscape that became TikTok during the 2019 Kashmir conflict. Bodies burned on the FYP and misinformation was rampant.
As Facebook witnessed with the entrance of Facebook Live, TikTok has also faced native suicides. The family of a Tamil Nadu mother of two tried to intervene in her TikTok addiction after her children were neglected, and she ultimately TikToked her suicide. indiatoday.in/india/story/ta…
That suicide was among the cascade of stories that flooded in as TikTok saturated India. They resulted in the first court injunction that banned the app. Among them were the Delhi murders of Mohit Mor and Salman Zakir:
The history of these platforms — and how they arrived to where we are today — is riddled with human suffering. For the small community that was Justintv (now Twitch), it was a 19 year old's suicide that had the brand name reaching headlines globally. wired.com/2008/11/teen-k…
"A computer user who claimed to have watched said that after swallowing some pills, Biggs went to sleep and appeared to be breathing for a few hours while others cracked jokes. Some members of his virtual audience encouraged him to do it" nbcnews.com/id/27855263#.X…
Like those that have come before, TikTok now has another crisis moment where it must decipher the role of the platform and its moderators. nytimes.com/2018/12/31/tec…
Given the amount of sharing/download metadata TikTok has, and how ByteDance casts its algorithmic genius, it's surprising that the company let this gruesome video through to the For You page. Zhang's exercises appear to have failed, and TikTok will be forced to evolve as a result
On the contrary to the FYP, TikTok's information architecture is quite robust, and has thus far responded to narcotics better than Instagram. (There are dark lessons from Douyin and the Chinese information control architecture that explain that.)
At this point, more popular than the suicide video are the TikTok creators informing viewers how to spot the video and report it. It's a unique cultural dynamic of TikTok, where the rapid turnover and lower bars to virality mean everyone plays in to steering current affairs.
Another ex of how TikTok users are competing with the video on the FYP plane. We also see this on Twitter, but a core difference is that TikTok has more control over what you see, rather than your defined network & actions. TikTok is closer to a broadcaster than Insta & Twitter
As long as humans and their struggles will exist, this will happen again. But, based on how TikTok users have responded to this, mobilizing other creators and their platform's elevator operators, I am given a bit of hope for how we can one day overcome these challenges.
A neat dynamic to study, sadly stemmed by something truly horrible. It's worth holding out hope that these platform lessons are helping the industry move in a brighter direction. We'll get there one day.
