Building a sophisticated troll network at scale had never been cheaper and easier—thanks to AI capabilities.
Even if you are not Russia, China or Iran, now you can influence Western countries towards far-right, populist parties (AfD, RN, PiS…) and isolating from each other.
1/
This is exactly what UAE* did: with AI-powered operation targeting 🇺🇸🇨🇦🇬🇧🇫🇷🇩🇪🇵🇱🇪🇸🇮🇹 and the Gulf states, it weaponized COP29, a UN climate change conference in Azerbaijan.
2300+ accounts in 8 languages pushing country-specific narratives with highly-realistic fake personas.
2/
With broad variety of pro- and anti- "opinions" in politics, economy, social, emerging markets, climate, all Western-targeting narratives agree on:
• desire to change of elected officials, established institutions and status quo
• UAE being the most successful alternative
3/
Across AZ/EN/FR/DE languages, Azerbaijani leadership and president Aliyev are endorsed for prosperity, stability and sovereignty.
In Nagorno-Karabakh conflict, Azerbaijan is portrayed as a victim of Armenia's aggression supported by biased, hypocritical France and the US.
4/
The powered-by-AI nature of this network was revealed by variety of glitches in LLM-generated tweet texts and GAN images.
It will take only a few generations of GPT models to get rid of artifacts like these. Meanwhile, some of our favorites in texts:
5/
GAN cityscapes are sometimes even more betraying, like:
• a wind farm in the middle of Paris
• double Gherkin skyscrapers in London
• two CN Towers in Toronto
• two Eiffel towers
6/
Most important evidence of inauthenticity and coordination for the network as a whole are:
• signature combination of creation date and first tweet across accounts
• broad variety of features that connect together all the 3 first-tweet-based cohorts
7/
• • •
Missing some Tweet in this thread? You can try to
force a refresh
🧵 In 3 days, pro-Kremlin fake-video operation received 3 debunking statements from FBI, totaling 1.5M views and counting.
This operation is active since at least Sept 2023. My team was first to report this operation back in Jan 2024, we codenamed it Matryoshka 🪆 (Russian Doll)
2/
Never before has Matryoshka gained that much attention from a govt agency, in the US or overseas, at the low cost of a dozen fake videos, which is literally half of its daily(!) capacity.
These statements alone gave so much airtime that it could not possibly dream of.
3/ This is what will possibly happen next:
💡 Matryoshka will be more often impersonating govt agencies than it was doing before, for as long as that keep resulting in public statements from those agencies.
🧵 To influence US elections with FAKE videos, Kremlin reused half a dozen ideas from its earlier campaigns targeting Macron, Paris Olympics and, of course, Ukrainian refugees and aid to Ukraine.
Some of them ⏬, based on our 10 months of tracking Matryoshka influence operation:
2/ (Keep in mind:
• all of those videos were fabricated, they were never published by a respective media outlet or govt agency
• most of these videos never got any traction. I am only showing them to expose disinfo methods of the Russian state, not to debunk those videos)
3/ Harris ⬆️ (Macron ⬇️) are showing signs of early Alzheimer:
♻️ the issues only appeared recently
♻️ their public communication demonstrates multiple symptoms
♻️ soon we will see further decline of cognitive abilities
@FBI 1. Both of the videos referred to by @FBI appeared on X/Twitter first posted by Matryoshka, a Russian state influence operation, which is ongoing since Sept 2023.
@FBI 2. The operation was later corroborated by @Viginum_Gouv, French agency monitoring and protecting against foreign digital interference (part of General Secretariat of Defense and National Security): sgdsn.gouv.fr/files/files/20…
@FBI @Viginum_Gouv 3. @CheckFirstHQ / @ResetDotTech codenamed effentially the same operation as Operation Overload:
1/ As the UK experiences its most significant riots since 2011, the Kremlin is happy to exploit the situation to its benefit by:
🔥 adding fuel to the fire of the #UKriots
🔥 inciting similar tensions in France¹⁶
🔥 weakening support for Ukraine¹⁷
🧵 Thread:
2/ Since Aug 7, Matryoshka, a pro-Kremlin network, uses bots to flood X/Twitter with fake visuals (videos, street art, and doctored covers) related to the Southport events.
The goal is to overload fact checkers with illegitimate reports and to boost virality of false narratives.
3/ Matryoshka uses fake visuals to better serve its goal. Compared to genuine news, they:
🎯have a greater disinfo impact if they go viral
🎯keep fact-checkers busy longer
That's why a visual promoted by Matryoshka is, by itself, a strong marker of its intentional false nature.