Paul Scharre Profile picture
Executive Vice President and Director of Studies at CNAS. Author of "Four Battlegrounds: Power in the Age of Artificial Intelligence."
3 subscribers
Mar 30, 2023 10 tweets 3 min read
@FLIxrisk has released an open letter calling for a moratorium on training large AI models more powerful than GPT-4.

Their specific proposal is vague and not very realistic, but it's a significant development nonetheless. [THREAD]

futureoflife.org/open-letter/pa… Large AI models like ChatGPT and GPT-4 are inherently dual use.

@OpenAI's GPT-4 system card walks through several possible misuse risks, including for hacking, disinformation, and proliferation of unconventional weapons (e.g., chem/bio). cdn.openai.com/papers/gpt-4-s…
Mar 16, 2023 4 tweets 1 min read
In the long run, detectors will fail.

Any methods of detection can be folded into the next generation of AI.

Fakes will converge towards reality. Watermarking will be key to distinguish fakes from reality.

Responsible actors will ensure their synthetic media is watermarked.

But not everyone will act responsibly. And generative AI is so widespread that there will be irresponsible actors.
Mar 9, 2023 11 tweets 2 min read
Data is a vital resource for machine learning. Does China have a data advantage?

Not so fast. China's alleged authoritarian advantage in data is overstated.

THREAD China's supposed data advantage comes from its massive population (1.4 billion people!) and rapidly growing surveillance state.

The CCP is building a surveillance system unparalleled in the world.

But that doesn't necessarily translate to a data advantage.
Feb 27, 2023 11 tweets 3 min read
China's model of AI-enabled repression is proliferating around the world, threatening human freedom.

Here's what the U.S. and other democratic nations can do to push back.
[THREAD]
latimes.com/opinion/story/… China is building a new model of tech-enabled authoritarianism at home.

The Chinese Communist Party has deployed 500 million surveillance cameras to monitor Chinese citizens.They increasingly use AI tools like facial and gait recognition.
Feb 14, 2023 15 tweets 4 min read
UFO jokes aside, I’m troubled that the U.S. military is shooting down aerial objects in U.S. airspace without positively ID’ing them first.

How long before they accidentally shoot down an aircraft? “We don’t know what it is; shoot it down” seems like a very loose ROE for domestic U.S. airspace in peacetime
Feb 14, 2023 10 tweets 3 min read
DARPA gave me incredible access for Army of None, but there was one program they stiff-armed me on:

TRACE, a DARPA program to use deep neural nets to improve automatic target recognition.

For Four Battlegrounds, I got the scoop! [THREAD] TRACE (Target Recognition and Adaptation in Contested Environments) was a DARPA program to improve automatic target recognition (ATR).

It was one of the first DoD programs to capitalize on the deep learning revolution. Image
Feb 8, 2023 5 tweets 10 min read
We went line-by-line through the new DoD policy on autonomous weapons so you don't have to!

The new CNAS Noteworthy on DoD Directive 3000.09, Autonomy in Weapons, analyzes changes from the prior policy and what they mean for the U.S. military
cnas.org/press/press-no… #CCWUN @RikeFranke @drfranksauer @WriteArthur @HMRoff @UNIDIR @icracnet @ODA_Geneva @AnjaKasp @tegmark @marywareham @FLIxrisk @JackieGSchneid @TobyWalsh @HaydnBelfield @brankamarijan @DaanKayser @PAXforpeace @Article36 @BanKillerRobots @elizabethminor3 @Lauren_A_Kahn
Feb 4, 2023 20 tweets 3 min read
One of the challenging things about international crises like the #balloon is how much ambiguity there can be about the drivers of a state’s actions.

We can see the balloon, but what does it *mean* about China’s intentions?

There are many plausible explanations:

THREAD (A) Intentional – Everything is going exactly as Xi intended. He wanted to agitate Washington and spike the Blinken visit. (Seems unlikely)
Jan 25, 2023 19 tweets 5 min read
The new, updated DoD Directive 3000.09, "Autonomy in Weapon Systems" is out.
esd.whs.mil/Portals/54/Doc…

Quick takes below on what's in and what's out. [THREAD] Bottom line: The DoD just released an updated Directive (an official policy document) that guides the U.S. military's policies on autonomy in weapons (e.g., lethal autonomous weapons).
Jan 25, 2023 22 tweets 3 min read
The All-Volunteer Force turns 50 this year. What does the war in Ukraine tell us about the future of the All-Volunteer Force (AVF)?

[THREAD] War is, fortunately, a rare occurrence. In peacetime, militaries build theories, implicitly and explicitly, of what future wars will look like that inform force design and force management.
Jan 23, 2023 24 tweets 5 min read
The United States has been on a steady path towards selectively decoupling U.S.-China tech ties. That's a mistake.

Decoupling alone will not secure U.S. interests. [THREAD] While overall U.S.-China trade ties are strong, U.S. policymakers have been steadily taking efforts to pull apart the deeply integrated U.S. and Chinese tech ecosystems.
bloomberg.com/news/articles/…
Jan 21, 2023 6 tweets 3 min read
I sometimes get a skeptical 🤔 response to concerns I've raised about countries falling into the trap of a "race to the bottom" on safety for military AI systems.

But it's worth pointing out that these competitive dynamics are happening *now* in the commercial sector. [THREAD] Excellent new reporting by @nicoagrant @nytimes on how Google "will 'recalibrate' the level of risk it is willing to take when releasing [AI] technology" in response to OpenAI's ChatGPT
(h/t @ProfNoahGian @ESYudkowsky)
nytimes.com/2023/01/20/tec…
Jan 21, 2023 12 tweets 3 min read
Legal battles are brewing over generative AI.

Artists are suing Stability AI and Midjourney, alleging copyright infringement for including copyrighted images in their training data without permission.

But this isn't the David-and-Goliath story you might think it is. [THREAD] At the core of the lawsuit is whether or not including copyrighted images in a training dataset is "fair use."

@jjvincent @verge breaks it down in this excellent series of articles:
theverge.com/2023/1/16/2355…
Jan 16, 2023 4 tweets 2 min read
News of rumored TikTok deal:
wsj.com/articles/tikto…

Good that it includes oversight of TikTok's recommendation algorithm.

Lots of questions about how one would police the algorithm in practice. CCP influence on TikTok's recommendation algorithm may not look like a backdoor into the algorithm, allowing Party officials to alter the code from China.

Lots of companies self-censor so they don't avoid run afoul of the Party by talking about "sensitive issues."
Jan 16, 2023 18 tweets 7 min read
Has China overtaken the United States in AI research?

Nikkei reports "China is the undisputed champion in artificial intelligence research papers ... far surpassing the U.S. in both quantity and quality."

But is that true?

[THREAD] The claim: A Nikkei study, working with Elsevier, reviewed AI academic and conference papers and found:

Quantity - China produced roughly 2x as many papers as the U.S. in 2021.

Quality - China had 70% more than the U.S. of the top 10% most-cited papers
asia.nikkei.com/Business/China…
Dec 14, 2022 14 tweets 5 min read
Just recently had a chance to dig into these videos and reporting by @Wyrwal over the summer on AI image classifiers being used on armed drones in Ukraine.

These are stunning developments. onet.pl/informacje/one… These are effectively turnkey lethal autonomous weapons.

saker.airforce
Oct 27, 2022 26 tweets 7 min read
1/ Washington is exuberant about the Biden Administration's new export controls on chips to China, but the new restrictions are likely to harm U.S. national security in the long run [THREAD] 2/ The new export controls use a variety of tools to target two key aspects of semiconductors:

1) China's domestic fabrication for high-end chips (16/14 nm and below for logic chips)

2) China's access to advanced chips, such as GPUs used for artificial intelligence
Oct 3, 2022 13 tweets 4 min read
Breaking news today on sweeping new U.S. government restrictions on chips to China coming soon.

Why these restrictions are:
(1) A big deal
&
(2) Likely a bad move

🧵
washingtonpost.com/national-secur… The chip ecosystem is highly globalized yet has key technology chokepoints that give a few countries (Taiwan, South Korea, the Netherlands, Japan, and the United States) outsized leverage over semiconductor fabrication. 2/
Oct 3, 2022 22 tweets 6 min read
The Biden Administration is working on a deal for government oversight of TikTok. The risk of U.S. persons' data being sent to China is real.

Yet far less attention has been paid to TikTok's recommendation algorithm and its insidious risks. [THREAD]

nytimes.com/2022/09/26/tec… 1/ What do we know about TikTok's algorithm? Recommendation algorithms play a powerful role in teeing up content for social media users.

Facebook and YouTube have come under fire from independent analysts for recommending extreme and incendiary content. nytimes.com/2018/03/10/opi…
Mar 12, 2022 22 tweets 3 min read
What lessons should we draw from Russia’s invasion of Ukraine for defending other U.S. partners (ahem, Taiwan) from autocratic aggressor states? 🧵 1. Imagine the unthinkable: U.S. policymakers need to expand their horizons of what is possible, pushing back on our cognitive bias to assume the future will be like the present.
Mar 11, 2022 12 tweets 2 min read
How can the U.S. stop Putin’s aggression in Europe without starting WWIII?

Summary 🧵 of my latest in @DefenseOne

defenseone.com/ideas/2022/03/… 1/ The greatest risk the United States and European countries face right now is mission creep from their own desire to do more in Ukraine. 2/