Thread by @joseph_fridman

Last talk of day 2: James Evans from KnowledgeLab talking to us @ #metascience2019 about “the social limits of scientific certainty.”

How can we scale up scientific logic to address All the Data? How does Science As a System think?

Evans: dense social connections distort certainty. Professional networks shape judgements in peer review; scientific validity is a sociological phenomenon. Clustered tastes are clustered epistemologies: centralized communities generate less replicable results. #metascience2019

Evans is moving very quickly through some complicated info visualizations. He’s doing cliometrics, discourse analysis, media reviews - but I am not sure at all about what these datasets are, or what specific methods there are for this. #metascience2019

Despite all the diff analyses and Evans moving quickly, the point remains super consistent: communication dependencies in tight-knit scientific communities cause issues with replicability. They do analyses on the Cochrane Databse of Systematic Reviews. Neat.

#metascience2019

Evans presents us with a trade-off - there are costs to being so tight-knit, so flocked, so herdlike; but there are consequences to distributing the work in unconnected groups. We’ll have to afford mistakes. (OMG plz don’t show an equation without explaining it) #metascience2019

“Scientific teams increasingly look like American football teams.” Larger, role-based. Bigger = better is becoming an assumption.

More than 1000 authors, 30+ years for 2017 Nobel Prize in Physics. 1 author in 1912.

(Read Galison? Here’s where STS comes in.)

#metascience2019

Like, if your slides are moving too fast for someone to take a photo, there’s no way in hell anyone can read a single sentence. #metascience2019

Evans analyzed 65m products (software on github, patents, etc.) and analyzed - well, shit, I’m not sure - how much things were connected to previous products, and their team compositions? Really pretty infoviz though.

#metascience2019

“Social connection and conditioning is associated with cultural collapse. Biological extinction; language on the interwebs; trade and economics.” Mutual information rises -> more joint entropy. This is, like, a mega-theory of all human interaction, civilization, and knowledge.

“This is a project where we tried to predict all of next year’s papers and all of next year’s patents.” They used keywords only, but: holy knowledge objects Batman. They predicted 97% of next year’s papers and patents. Inertia.

#metascience2019

So: best predictor of success from outsiders is being unpredictable. (Is this confirmation bias? I can’t tell!)

Other work looks at success of grants, startups, terrorists - a connection between persisting and not overlearning from failure and success.

#metascience2019

There’s a Shannon communication diagram in here? He’s analyzing polysemy? 30m papers analyzed: more ambiguous words create fields for social learning. (NLP for paradigm shift?)

There’s also polarization Wikipedia analysis here. (H/t to @SimonDeDeo.)

#metascience2019

OMG. Evans: there’s a trade-off between what modernity does and what modernity wants. Acceleration kills diversity, which is what you need for good acceleration. Bridging but independence.

(Is this the dialectic?)
(I picked the wrong day to stop sniffing glue.)
#metascience2019

OH HEY WAIT I KNOW THIS ONE! IT’S FUCKIN’

SEEING LIKE A STATE BY JAMES C. SCOTT

BOOM