Thread by @EliotHiggins

🧵 Important investigation by @mariannaspring on how social media algorithms push harmful content to young users. This connects closely to my research on online radicalisation. Let me explain how.

https://twitter.com/BBCWorld/status/1830497450571845976

What happened to Cai in this article is a clear example of how online radicalisation often begins. It starts with seemingly harmless content and quickly escalates because algorithms prioritize engagement over user safety.

Social media platforms use algorithms designed to keep users engaged by feeding them engaging, and sensational, content. This means a teenager watching a few neutral videos can suddenly find themselves immersed in more extreme or harmful material.

The issue isn’t just about the content itself—it’s about the pathways these algorithms create. Once a user engages with content that the algorithm finds "interesting," it starts recommending similar or even more extreme material, regardless of whether it’s safe or harmful.

This is where the concept of the "pyramid of radicalisation" comes into play. At the base of the pyramid, you have the widest number of users who are exposed to mildly provocative or neutral content. A percentage of these users will engage with more extreme content as they encounter it.

This is where the concept of the "pyramid of radicalisation" comes into play. At the base of the pyramid, you have the widest number of users who are exposed to mildly provocative or neutral content. A percentage of these users will engage with more extreme content as they encounter it.

The danger is that those at the tip of the pyramid may feel compelled to take real-world action, which can include violence. Imagine someone who starts by googling "are vaccines safe" and ends up burning down 5G towers because they think they’ll activate the microchips Bill Gates supposedly put in vaccines. This is the kind of real-world impact that can result from online radicalisation.

But it’s not just the content that leads to radicalization—it’s also the communities that form around this content. As users consume more extreme material, they often find themselves in online groups that reinforce and amplify these views, creating a powerful feedback loop.

These communities provide a sense of belonging and validation, which can be very appealing, especially to those who feel alienated or distrustful of mainstream narratives. This is where radicalisation really takes hold—through community and interaction, not just through content.

The algorithms act as gatekeepers, deciding what content and communities users are exposed to. In doing so, they can unintentionally guide young users down paths toward more radical thinking by constantly feeding them content that fuels outrage or excitement.

Cai's experience, as detailed in the article, is a perfect example of this phenomenon. He started with harmless videos and soon found himself overwhelmed by violent, misogynistic content, leading to increased exposure to harmful ideologies.

To tackle this, we need a comprehensive approach: better transparency around how algorithms work, more effective content moderation, and education that empowers users to critically evaluate the content they see online.

Radicalisation online isn’t just about the content—it’s about the digital environment that platforms create, where extreme views can spread unchecked. This is why understanding and addressing the role of algorithms and online communities is so important.

Ultimately, it’s about building resilience and awareness. We need to teach young people—and all users—how to navigate digital spaces critically and recognize when they are being manipulated by algorithms designed for engagement, not safety.

Thanks to @mariannaspring for shedding light on this critical issue. It’s up to all of us to ensure social media platforms prioritise user well-being over engagement metrics