Thread by @perryworldhouse

We're now starting coverage of our conversation with @ODNIgov's Morgan Muir, moderated by @mchorowitz. Follow this thread for updates 🧵

"We have an eye toward longer-term things we need to do to advance American national security interests, to deal with challenges the world faces. We focus on what we are assessing, why we think what we think, how to capture uncertainty as we talk about opportunities," says Muir.

"We use a range of qualitative and quantitative methods to do this. We work with experts in the field, in the academic community; we employ analytic techniques to challenge assumptions. One question is, how do we augment human analysis and expert judgment with machine learning?"

"The tools have to work for the analyst and better serve the policymaker. We need to demonstrate to both the benefits of blending machine-based analysis with expertise and delivering that insight at speed. Often, we need the answer to a question that day, and it's a challenge."

"The latest Global Trends report was called ‘A More Contested World,'' says @mchorowitz. "How do you put those reports together?"

"The Global Trends report comes out every four years, normally when a new admin comes into office, with the idea of looking back at the past for the trends we missed and looking ahead 20 years. We start thinking about that theme about a year before, working with outside experts."

Click here to learn more about the Global Trends report: dni.gov/index.php/gt20…

"What has the intelligence community gotten right?" asks @mchorowitz. "It’s easy to frame something so broadly it’s impossible to be wrong," says Muir. "We need to frame pieces in ways that convey opportunity and risk to the policy side, and we’re not always going to be right."

"We once did a piece on counter-narcotics, looking out 4-5 years. One of our seniors at CIA told us that we had essentially been too vague in our assessment. We'd said country X will make progress on counter-narcotics, but not solve the problem, within a wide range of 5 to 98%."

"If you're a policymaker, you want to hear, "you’re at 20, but you could get to 30 or 40, and here are the options to get there." If you get into that space, the analysis is much more relevant. But that’s a harder thing to do."

If more precision would be better for policymakers, why has that been hard to institute in the intelligence community?

"There are a lot of challenges. Understanding what’s being asked, getting someone to own it. But the main issue is – how much of an improvement does it offer, how consistent is that improvement, and is that improvement worth the cost of getting there?"

How do analysts judge what they've gotten right or wrong?

"With Iraq and WMD, there was an effort to look at what happened there, and those insights were factored into future IC training," says Muir. "New analysts coming in were taught about the mistakes that were made and things to be aware of."

"What I would tell analysts as a line manager is, go back 3 months later, look at what you’ve written, and how you'd change it. Partly you’re smarter as time passes, have new data, but it’s understanding what you could have done better and applying it to what you’re writing now."

"We get feedback from policy, they let us know how we’re doing in addressing the questions they had, whether they find products useful. I think there's a robust effort to refresh how we think about issues, and how we inculcate lessons learned into training."

"Open source intelligence has not long been a focus for intel community, but some groups like @bellingcat have harnessed this in an impressive way. Could we do a better job of exploiting publicly available info?" asks @mchorowitz.

".@ODNIgov is looking at open-source efforts right now. As we look around the world, every Fortune500 has an open-source element that’s trying to better understand the environment they’re working in. The intelligence community needs to better tap into that moving forward."

"But open-source collection is a lengthy process, we’re working on what a longer-term approach to leveraging open-source looks like. But it’s also resources – how do you prioritize what you want out of it when it’s made up of so many different things?"

"There's been a recent push for greater transparency from the intelligence community," says @mchorowitz. "How is @ODNIgov approaching that?"

"We recently put out an unclassified summary of key judgments on COVID-19. We’re looking at ways to, with the National Intelligence Council, publicize more of its work and key judgements, so the public has a better idea of what we're focusing on and how we come to conclusions."

You can read the recent COVID-19 report here: dni.gov/index.php/news…

"How do you think about working with allies and partners, especially learning best practices from each other? I'm thinking particularly about the UK's Cosmic Bazaar," says @mchorowitz.

"We engage a lot on the analytics side, and I think we’re going to be doing more in the years ahead to collaborate more and leverage what’s out there. In a place like London, they approach these challenges in different ways," says Muir.

"How do you stumble upon new ideas and approaches that challenge the way you think? There’s limited value in engaging with a partner who’s going to tell you something you already know. It’s a much better discussion with someone who’s come to a different conclusion."

"What advice would you offer @Penn students thinking about joining the intelligence community?" asks @mchorowitz.

"We once hired somebody who had OK grades, but in their interview, spent a lot of time talking about working in their dad’s grocery store and what they learned. It’s someone who has a life outside of their field of expertise, is empathetic, understands give and take."

"We want the person who is curious - you’re more willing to try new things, experiment, think about different approaches. We also want someone who wants to make a difference. We have these enormous challenges, and you can help inform the President of the U.S. about these issues."

Thanks for following our coverage of today's event! Don't miss tomorrow's conversation with @MaEllenSirleaf, former President of Liberia. Register here: global.upenn.edu/perryworldhous…