“This paper address the threat of automated influence - purposeful use of AI to monitor specific audiences and produce and distribute misleading information to them over digital media networks for the purposes of foreign security objectives”
“Three particular capabilities enable this new thread factor: algorithmic content generation, personalized targeting and firehose dissemination.“
“Adding machine learning to info operations allows users to micro-target the audiences most susceptible to the latest behavioral psychology techniques, to exploit emotion and bias and to concentrate on those target groups that are best placed to affect the desired outcome.“
“With that guidance, customized mental munitions can be fired machine gun speed.“
"Even if malign actors can lie at machine speed, they still Have to get the story to an audience; personalized targeting is required for the Influence Machine to know who will except a particular message."
"The uniqueness of individually lived experiences should decentralize the inputs to an unknowable variability, but the monetization of social science-at the nexus of tech stocks and marketing fees-has rendered many human behaviors quite predictable for recurrent neural networks."
“If the Influence Machine knows whom to target and to what they will respond, its next prediction is when to provide catalytic input. The answer is: always, in firehouse fashion.”
“If ‘repetition is a key tenet of IO execution,’ then this ‘machine gun’ ability to fire information at an audience at will, with faux-empathetic precision and custom content, provide the means to change a decisive audience’s very reality.”
Is there precedent for the Influence Machine?
“The Protestant reformers in Europe vastly out-printed their Catholic opponents in the critical years from 1521 to 1526”
“The Protestant reformers in Europe vastly out-printed their Catholic opponents in the critical years from 1521 to 1526”
“Printers in at least 13 towns were producing material for Luther... Because of the decentralized production, rapid distribution and consumable content, the ideas of the Reformation overwhelmed local officials”
“The modern proliferation of social media influence clearly resembles the scale of technology diffusion of the Reformation, but with capabilities for content production and dissemination that are multiplied by several orders of magnitude. “
“Reporting on Russian operations in 2016... suggests accounts operated by Putin’s digital agents generated over 60 real protests... they tapped into the protest potential of a local population without setting foot in the theater.”
“The Influence Machine is most useful to manipulate a certain value sentiment, a certain selectorate, for a certain period of time and in support of some specific policy goal, with negative consequences for the opponent.“
“Today’s leading Russian tactician- General Valery Gerasimov- believes that ‘the role of non-military means of achieving political and strategic goals has grown, and in many cases, they have exceeded the power of force of weapons in their effectiveness.’”
LINEAR STRATEGY IN AN EXPONENTIAL WORLD
"Though the Pentagon's strategies recognize the proground potential of AI and call for increased focus on advanced computing, the application of AI to drones, especially small ones, tends to get the majority of the attention."
Drones aren't the big deal though... "Whether armed with directed energy weapons or powered by hypersonics, robots in a flat world are limited by classical mechanics and are similarly limited in their strategic impact."
"Controlling killer drones through combinatorial optimization problems in fluid multi-agent systems is only one application of AI in military operations. Situational awareness and decision-making are arguably where the DoD is making the most progress..."
"Though terrorist drones seem to present a significant threat and "Siri-for-Soldiers" appears groundbreaking, AI-guided robots and mission command optimization present only linear change, whereas automated influence has exponential potential."
"The distribution of slaugherbots can occur at about 25 miles per hour - the distribution of mems on the digital battlefield occurs at close to the speed of light."
"Dmitry Kiselyev, the director of the government-controlled news agency Rossiya Segodnya, maintains that, today, it is much [costlier] to kill one enemy soldier than during World War II, World War I, or in the Middle Ages; if you can persuade a person, you don't need to kill him"
"Like strategic bombing of generations past, the Influence Machine aims at massive strikes deep into the state... but unlike strategic bombing, the destructive event does not create a shared experience. Instead, the goal is to divide at a personal and tribal level..."
"Gravitation toward drone strategies is reminiscent of an age of 'thermodynamic warfare,' in which the ballistic energy needs of ever more physically destructive weapons drive the economic mobilization for total war."
Attraction to machines learning tools that optimize military informational processes is evocative of 'cybernetic warfare' driven by the fantasy of omniscient knowledge of the battlefield, a dream that should have ended in the jungles of Vietnam""
"As part of its adaptation for an age of 'chaoplexity' [a time of positive feedback loops + non-linearity], the USG must confront the cognitive influence potential of these technologies in three key ways: funding... designating an authoritative body... and innovative techniques"
"There are quite a few disparate efforts underway to identify and grapple with the threat of automated influence..."
- Oxford's Computational Propaganda project
- German Marshall Fund's Hamilton68 database of Russian Twitter influence
- Oxford's Computational Propaganda project
- German Marshall Fund's Hamilton68 database of Russian Twitter influence
DARPA has worked on identifying deep fakes, but only got to 40% effectiveness and "even this meager gain is offset by malign actors' constant innovations that make them increasingly difficult to spot."
"Perpetrators can always lie faster than any debunking process can operate"
"...the Kremlin uses a high number of channels and a dizzying array of messages to overwhelm... When this Russian firehose turns on, the stickers and posters that fueled the last generation of political arguments will not be enough to preserve Western cognitive integrity."
"The Pentagon is spending $7 billion a year for robotics and practically none on automated influence, all while Russia operates a $1.3 billion state media apparatus."
"The Russian government as integrated a comprehensive concept for maneuvering within a 'unified information space'... Many have argued to restore a U.S. Information Agency... The potential political backlash must simply be stomached."
"Competitors must be beaten at scale with truthful content."
"The U.S. military must curate faster than its opponents can lie... The DoD needs radical transparency, though certainly with a selective eye; existing AI capabilities can provide it."
"There is a duopolistic market, with significant first-mover advantage, for information coming from the front lines... The DoD must develop ways to exploit that sort of data. This combat footage could be a powerful and effective tool, but is instead bring wasted"
"One must ask, what had greater impact on U.S. strategic options in the last half of the 20th century: the antitank guided missile, Airborne Warning and Control Systems or the advent of 24-hour cable news?"
"Private industry will continue to refine the automatic weapons of influence described above; commercial attempts to profit from the shortcomings of human cognition with ever-greater computing power and connectedness will drive this arms race."
@threadreaderapp unroll
