Thread by @davetroy

1/🧵For the last few years I have been working on some core ideas underlying misinfo/disinfo problem. I've been wanting to share this in an essay or book form, and have not yet had time to do so. But because this is urgent and keeps coming up, I want to share the crux now.

2/Hypothesis: social capital is foundational to society; its manipulation is a form of cultural/social engineering; misinfo/disinfo alters social capital; pathological social capital forms are a root cause of our dysfunction; we must promote 'healthier' social capital formation.

3/What is social capital? A bond of trust between any two people or entities. That tie might be strong (family, IRL friends), weak (social media), or parasocial (to media entities). Or somewhere in between.

4/A network of social capital formations can be considered as a whole, such as in this article. We see two distinct "partisan" networks, with some limited cross-cutting ties between them. Each of these networks is susceptible to confirmation bias, etc. misinforeview.hks.harvard.edu/article/right-…

5/One way to define a cult is as a network of social capital that is isolated from other networks; such a group tends to put its own 'local' interests ahead of any 'global' concerns. It may or may not have a single charismatic leader, but it lacks for ties to other networks.

6/Such 'cultish' networks may include: religious cults, personal growth cults, mafias, groups of rogue traders, social phenomena such as QAnon, rogue political parties, gangs, criminal networks. Pathology is defined by 1) too-inbred social capital, 2) tendency to violate norms.

7/We can also conceive of 'negative social capital': the 'repulsion' and distrust that one entity may feel towards another. Networks with hardened in-group/out-group boundaries have negative social capital towards each other. This can be self-reinforcing (i.e. GOP vs. HRC).

8/Belief in and assessment of what constitutes 'truth' is a social process. People within a tightly meshed network tend towards shared truth. There is thus social pressure to subscribe towards that truth, or risk social ostracism.

9/Group membership is also a basis for social identity; people don't want to reject their own identity. For those interested in this set of phenomena I recommend 'Linked' by @barabasi and 'Connected' by @NAChristakis + James Fowler; many other academic papers on this subject.

10/Disinfo/misinfo tends to divide populations into smaller and more distinct networks with more internal social capital and fewer cross-cutting social ties. By forcing people to 'take sides' on a multitude of issues, groups become more hardened and more tightly wound.

11/Political scientists @LilyMasonPhD and @NathanKalmoe have found that the more tightly wound these networks are, without cross-cutting ties to other networks, the more reactive and prone to political violence they are. This is how disinfo works to make society more violent.

12/While we have tended to view misinfo/disinfo as a first-order problem based on truth/falsity, this is less valid the more you look at second order effects. And even 'true' information, properly timed and directed, can also have negative effects on social capital formations.

13/Those familiar with Claude Shannon and information theory might consider that 'information' doesn't just reduce uncertainty; it should also be considered in terms of its effects on human + networked targets. Info must be considered in terms of effects; not just true/false.

14/Therefore, ethical decisions about what constitutes useful media should not be restricted to considerations about accuracy, but also must include intended and actual effects on target populations. Propagandists know this all too well.

15/We also tend to sandbox the misinfo/disinfo problem as a "media" or internet/tech/social media issue; but in fact if the mechanism of action is social capital, we need to consider a whole of society approach to promoting healthy configurations of social capital.

16/One reason the US is susceptible to social capital manipulation strategies is because we are a very large country that's mostly empty. The framers attempted to manage this using the best in 18th century philosophy and technology, but the core vulnerability remains.

17/Our biggest divide is urban vs. rural, and the cross-cutting social capital across that divide has always been lacking. But it's a very easily accessible division today especially as it coincides with many other attributes, as Mason has found.

18/In the end, we need to decide what constitutes a "healthy" United States social capital configuration, and we need to find ways to measure that at scale and manage our way towards it. This must become our primary domestic national security priority.

19/Achieving this is more than a media/news/internet/social media problem. It requires a whole-of-society approach to understanding social capital network formation, and curbing those who aim to destroy social capital as part of social engineering efforts.

20/What might we do? National service programs, promoting more mixing in educational settings, building bridges between religious networks, promoting cross-network mobility; considering effects of media on social capital. There are many, many pathways towards better outcomes.

21/In physics, we didn't really understand gravity until we considered how masses act on each other. I believe the same is true with social capital; we need a better concept of 'social gravity,' and of how it is manipulated by social engineering using targeted information.

22/While some have speculated that we live in a 'post truth' era, I disagree and find the notion both false and unhelpful. Truth objectively exists. What we face now is a world where cultish networks with pathological social capital reject truth for their own purposes.

23/Steve Hassan, @CultExpert, says instead that we live in an 'age of influence,' and I believe this is the correct frame. Influence, when abused to the point of creating pathological social capital, can cause people to reject objective truth.

24/As Daniel Patrick Moynihan famously said, “You are entitled to your opinion. But you are not entitled to your own facts.” Logically, it follows that people then are not fully entitled to their own private realities that hamper the ability to counter shared threats.

25/The retreat into private facts and private realities is one obstacle in dealing with threats like climate, income inequality, and many more. It's long past time for us to understand how misinfo/disinfo drives pathological social capital and the formation of cultish networks.

26/The reason why we keep hearing about people who have "lost family and friends" to QAnon and the like is because people are trading their strong ties for a community of weak ones, and identity that comes with it. One kind of social capital is destroyed; another is created.

27/The point of such an operation isn't to push 'false beliefs' so much as to rewire social networks and break down society as a whole. This is true for most disinfo; it's not about the belief, it's about the social consequences of harboring/espousing a belief.

28/It is important to consider social capital network formations as separate from 'politics' per se; cultural/social engineering has political consequences, but they are not the same thing. Gramsci/Breitbart/Bannon captured this: "politics is downstream from culture."

29/Trying to effect political outcomes without cultural engineering, if your opponent is engaged in cultural engineering, is a losing proposition. We must all engage in it to gain or maintain a political advantage. And we must criminalize unethical manipulation.

More on definitions of social capital here:
socialcapitalresearch.com/literature/def…

30/For those familiar with network analysis: people, media entities, concepts, outlets can be considered as potential 'nodes' in a network, while social capital is the 'edges' (relationships + weights) between the nodes. Because it's "unseen," we tend not to measure it.

31/Historically it's sort of messy to measure network connections. Sandy Pentland has innovated some clever ways to measure it in workplace settings, but we are really dealing with measuring something real but unwieldy we have ignored previously.
amazon.com/dp/B00DMCUYRM/…

BTW, this is consistent with so-called "hybrid warfare" concepts of "reflexive control." If a target network is sufficiently tightly wound and responsive to stimuli, it can be provoked into reacting in predictable ways. This can be a valuable part of any warfare strategy.

Here's a decent piece on Reflexive Control.
georgetownsecuritystudiesreview.org/2017/02/01/dis…

32/A few folks asked about further definitions of social capital. There are some here, and there are many academic papers that discuss both positive and negative forms of social capital, in various contexts.
socialcapitalresearch.com/literature/def…

33/We need to reframe the misinfo/disinfo problem as a multidisciplinary one that engages sociologists, psychologists, and behavioral scientists in addition to people from media, tech, politics, military/defense, security, and intelligence worlds.

34/I will write more on this soon, using this thread and your input as an outline. But I wanted to get this off my chest, as I think it's a useful framework for a broader, more useful lens onto multiple aspects of this problem. I appreciate everyone's insights. 🙏

35/Some works cited or referenced, for further reading... Uncivil Agreement: How Politics Became Our Identity, by Lilliana Mason
goodreads.com/book/show/3610…

36/Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life, by A.L. Barabasi
goodreads.com/book/show/1542…

37/Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives, by Nick
goodreads.com/book/show/6494…

38/The Upswing: How America Came Together a Century Ago and How We Can Do It Again, by Robert Putnam and Shaylyn Romney Garrett
goodreads.com/book/show/5012…

... Christakis and James Fowler (ack, Twitter editor ate my tweet!)
goodreads.com/book/show/6494…

39/Steve Hassan's BITE Model of Authoritarian Control correlates well with presentations of pathological social capital in real-world networks. His recent work to provide legal definitions for undue influence is also hugely useful.
freedomofmind.com/cult-mind-cont…

40/Our bias towards 'American Exceptionalism' and individualism blinds us to these manipulations; each of us thinks we cannot be manipulated, but really we are misdirected from the true target — society as a whole. We simply don't see it, like the very best magic tricks.

41/A couple of other thoughts: to the extent we “know when we see” healthy cross-connected networks as well as unhealthy pathological cultish networks, it follows there is a mathematically definable crossover point between the two. How can we measure + stay on the “healthy” side?

42/It also follows that there is an ethically optimal network configuration that can be pursued separate from ideology or politics. Ideal network topology is arguably a global good; at minimum it can be argued one can alter networks without consideration of ideology frameworks.

43/We can also assert that there are pathological network configurations that are incompatible with what we think of as “democracy”; excess undue influence poisons it. Democracy therefore requires some ethically normative network properties we have, as yet, failed to articulate.

44/Thus, this framework can help us evaluate any process that affects social capital creation or destruction, from media to lobbying, to disinfo, to consider whether it corrodes or enhances the function of democracy.

45/Likewise we may wish to allow some corrosive properties in order to avoid settling into an overly staid or monotonous mode that may suppress new ideas or innovations. There is an evolutionary purpose to occasional offshoots from the pack, if they are rooted in reality.

46/This book, Useful Delusions, outlines such evolutionary benefit to our tendency to break away and coalesce around new ideas, and we don’t want to completely suppress that, risking settling into groupthink. amazon.com/Invisible-City…

47/So this further hints at the idea of a network topology “sweet spot,” where we are all just connected enough to trust and respect one another to some degree, but not so meshed that our thoughts are homogeneous or lack diversity. This “just right” zone is hard to maintain.

48/So what sort of timeframe are we looking at here? Strong social ties don’t come about overnight. Supposing a major investment in something like national service, we might yield returns in 5-10 years, but really this is a 30-50 year, permanent strategic shift in direction.

49/I mention national service because it is one possible intervention; there are many others that might yield more localized benefits, or sooner. In all cases we need to consider effects on social capital configurations and how or whether interventions help bridge divisions.

50/This leads us to the sticky, normative question of what constitutes a healthy network, compatible with democracy? We might offer, 1) should minimize cultish, harmful networks, 2) should promote acceptance of facts, 3) should be compatible with scientific method.

51/Yes, people will dispute each of those points, but a majority can likely agree on those general principles. We can also likely agree on what we don’t want, and that’s authoritarian control ensconced in network topology. That may come in multiple forms.

52/Robert Jay Lifton articulated the properties of “totalism” in China, which tends to weed out dissenters. “Fascist” network topologies rooted in divide-and-conquer strategies are likewise incompatible with democracy.

goodreads.com/book/show/5492…

53/So the “biozone” for a network to support healthy democracy turns out to have fairly well defined guardrails: not too divided as to be fascist, not too monolithic as to be totalitarian. Information should flow freely; people should have freedom to move between subnetworks.

54/So who should own this problem? IMHO it is our biggest national (and global) security threat and should be paid for from the defense budget. But that doesn’t mean it should live in the Pentagon; it’s a public health and civic concern, upstream of defense consequences.

55/Every $ spent on civic health likely yields manifold returns in defense costs, climate mitigation, healthcare, and global stability. This needs broad involvement from exec branch, Congress, Interior Dept, Public Health Service, Surgeon General, military, etc…

56/Unfortunately nothing about this is terribly easy, but I think it at least offers a framework for breaking down what seems like an intractable, terminal condition into something we can talk about and potentially manage, which is a good first step.

Another interesting angle here is the research @johannhari101 has done on addiction and depression; I believe he found this same network pathology I describe here, and that the same social disorders make us vulnerable to addiction, depression, disinfo and social manipulation.

57/An eventual goal might be to achieve a level of cross-cutting resilience such that cultural/network engineering attacks are too expensive and low-yield to consider. But right now we are a highly accessible, cheap, reactive target for bad actors, both domestic and foreign.

58/Perhaps notable: we didn’t talk once about education, intelligence, or media literacy. That was on purpose. In general we overemphasize these factors, I think; network pathologies override them. Otherwise “smart” people can fall into network black holes they can’t escape.

59/And to return to the “gravity” analogy, social capital is to gravity as cults are to black holes. And we are all susceptible, in varying degrees, to some kind of social manipulation.

Likewise, some have joked: “We used to think, ‘if only people had access to more information.’ Clearly that wasn’t the problem!” Indeed. Social capital pathologies are the problem, and people used access to info to reinforce them!

And we can also add this corollary: “If you want to change someone’s mind, start by changing their friends.”