Twilight of the Human Hacker — how the Pentagon is turning to AI to be faster than humans in cyberwar — a thread on my story for @publicintegrity co-published with @yahoonews… 1/24
@publicintegrity @YahooNews If you’re just tuning in this is the latest installment in our #ScaryFast series on the future of warfare through the people and technologies that will shape it. The last feature looked at AI in weapons systems and was published with @TheAtlantic… 2/24
@publicintegrity @YahooNews @TheAtlantic The overarching issue that’s driving these new technologies is the need for speed (sorry). Increases in computing power are reducing the decision time in nearly every domain of warfare. Missiles are faster, computers can process more, and humans are becoming the weak link 3/24
@publicintegrity @YahooNews @TheAtlantic In cyberwarfare, the focus of this latest feature, that means building and parrying cyberattacks happens at superhuman speeds, and the methodical largely analog means that cyber experts have used to wage war are unlikely to keep up 4/24
@publicintegrity @YahooNews @TheAtlantic That’s probably the biggest surprise for most people in that until very recently, most cyber operations were planned and executed using big sheets of paper or white boards and methodically planned pieces of attacks carried out on disparate computers 5/24
@publicintegrity @YahooNews @TheAtlantic Every cyber unit has at least one story of somebody accidentally leaning against a white board while half asleep and wiping out a battle plan. Individual steps were often tracked by slips of paper piled on a central desk as troops carried out parts of orders 6/24
@publicintegrity @YahooNews @TheAtlantic That’s where @DARPA Plan X comes in. Looking at that environment, DARPA decided it would try to develop a new way of waging cyberwar that would be more computerized. The mantra at first for the developers was they wanted hand gestures, not keyboards, to drive the thing 7/24
@publicintegrity @YahooNews @TheAtlantic @DARPA What they developed at first wasn’t all that useful, but looked really cool. That’s when military planners especially at US Army Cyber started to shape the program. The key thing here is that cyber is very different for DARPA than pretty much any other area of war 8/24
@publicintegrity @YahooNews @TheAtlantic @DARPA Normally DARPA is working on some basic research type tools that are unlikely to see the battlefield for decades, a tribute to their crazy scientist reputation. But cyber moves so quickly that the tools they develop are rapidly ushered up to the NSA and Cybercom 9/24
As one DARPA official told me, the thirst for advances in cyber is so great prototypes get quickly snatched up for military and intel applications. And cyber is also different in that there’s no legal restriction on using AI or machine learning for a lot of operations 10/24
The former DASD for cyber said to me that he didn’t know of any restrictions on AI in cyber. Part of that is most cyber combat is really about stealing information, and espionage is widely unregulated legally. That’s intentional. No country wants limits on spying 11/24
Where things get a bit different is when cyber tools could have disruptive impacts on systems, like the Stuxnet attack. Once you start blowing up centrifuges you’re not engaging in espionage anymore, and with more systems networked the opportunities have blossomed 12/24
Stuxnet created a lot of handwringing in government circles because while it was successful, it got out and the public became aware of the cyberattack. It only reinforced the idea that code could run awry and maybe shut the power off in a hospital or something 13/24
Military leaders convinced DARPA to focus on using the ML in improving Plan X to help predict the probability of success for an attack, and gauge the potential for collateral damage. If you know those factors, you’re more comfortable saying yes to a strike 14/24
They also wanted to see if more of the decision making could be automated to allow strikes with minimal additional programing. That was partially inspired by the DARPA Cyber Grand Challenge, which pitted computers against each other in a capture the flag competition 15/24
“When I saw that, I’m like, ‘Oh my gosh, if you put that together with Plan X, this is how you would conduct operations,” retired Lieutenant General Ed Cardon, who at the time was running US Army Cyber, told me 16/24
What you end up with once you throw the predictive capacity for planning attacks and the ability to automate aspects of cyber combat is Project Ike, the direct descendant of Plan X. The program was renamed because of the original sci-fi association with the DARPA program 17/24
The thing about Ike is, once you have the predictive elements in play, and you can automate attacks, there no reason you can’t (and some say should) just let the computer launch attacks when it thinks they’re above a certain threshold of probable success (say 95%) 18/24
It would make your operations infinitely faster, and allow you to destroy aggressor systems before a human could possible take action, helping you defend your network. The flip side is you’re reliant on the ML system making good decisions and that’s tough. 19/24
Quoting @BuchananBen from the piece, “Machine learning is often like a smart but lazy eighth grader taking a math test: It’s great at getting the right answer, but often pretty bad at showing its work.” The system just can’t explain its decision 20/24
@BuchananBen The big question for policy makers now is less about technology, and more about are you willing to get the speed savings from ML in a system like Ike with the risk of something reacting in an unanticipated and dangerous way. 21/24
@BuchananBen Reducing reaction time, as this new generation of weapons is doing, has made military planners increasingly comfortable with that risk as they don’t want to get surpassed by competitors, but it’s not a discussion that has been carried out in any definitive way as of yet 22/24
@BuchananBen And AI systems cheat (or at least use shortcuts permissible under the rules they’re told but logically not doable for humans). Check out this piece I did on how an AI system swept a competition against a human pilot… 23/24
@BuchananBen Want to learn more about the ethics questions posed by AI in the military? Stay tuned. It’s something I’m continuing to report on in depth. Special thanks to the @DARPA folks I talked to for the piece who weren’t quoted, cutting room floor and all. 24/24

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Zachary Fryer-Biggs

Zachary Fryer-Biggs Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @ZachFB

21 Aug
A couple of observations from watching parts of three days of this @DARPA competition, where AI algorithms serving as fighter pilots in simulated dogfights squared off. (thread)
@DARPA First off, the AI system that won was definitely engaging in the kind of reward hacking behavior we see when algorithms try to find an edge in games. It was basically committing to kamikaze runs, charging its opponent to within 100 ft while locked in and firing.
@DARPA This is mentioned in the story, but that kind of flight behavior would be really risky for a real world pilot, as he or she could fly right through a debris field created by their slain foe. It's also against the rules in any normal dogfighting contest.
Read 16 tweets
10 Jun
I try to avoid getting too personal but as the great (x) grandson of a confederate military officer, let me offer a couple of thoughts here most importantly — I don't find it in any way diminishing to my family or history to remove these names from bases and/or monuments (thread)
Like many descendants of those who fought for the confederate side, I have a complicated relationship with our history. It's important to not turn a blind eye to it and my family has held on to some personal effects from my great (x) grandfather including his journal from the war
He had been in an infantry unit until the siege at Vicksburg, and after surrendering signed his "parole" which stated he'd never take up arms again against the US government. He immediately violated that when his unit reformed as a cavalry unit.
Read 13 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!