“Normalization of deviance,” a conceptual framework due to organizational sociologist Diane Vaughan, might help understand the current shaky state of science.
sma.nasa.gov/docs/default-s…
sma.nasa.gov/docs/default-s…
Deviance from systematic error-prevention rules gets normalized when a community of practice repeatedly recovers from near-disasters. That gives the impression that the rules were unreasonably strict, and you can walk closer to the line in future… gnssn.iaea.org/NSNI/SC/TM_ITO…
Management pressures front-line workers to do more. Feeling overworked, they want to expend less effort where feasible. Safety checks are easy to jettison because, nearly always, nothing really bad happens.
gnssn.iaea.org/NSNI/SC/TM_ITO…
gnssn.iaea.org/NSNI/SC/TM_ITO…
Management—in the science analogy, the senior figures in the field who edit the journals and make the funding decisions—justifies what would previously have been considered deviance using group-think strategies, “for the benefit of the field overall.” sma.nasa.gov/docs/default-s…
Once deviance is normalized, most participants don’t even notice it. They learned the notional rules, but those are obviously and routinely ignored, without any negative consequences, so they live in a separate mental compartment.
Defending against system failures (including those revealed in the science replication crisis) is feasible only when it’s part of everyone’s job.
how.complexsystems.fail
how.complexsystems.fail
Individual failures are visible only to people actually doing the work—not to higher-ups. Scientists in the trenches know most journal articles they see are nonsense (and must find informal ways of weeding them out). Editors measure citations instead gnssn.iaea.org/NSNI/SC/TM_ITO…
The work of reform, to make a system resilient to sources of error, is necessarily meta-rational: working on the system from outside.
It necessarily depends on circumrational understanding: how the system interacts with its non-systematic environment.
how.complexsystems.fail
It necessarily depends on circumrational understanding: how the system interacts with its non-systematic environment.
how.complexsystems.fail
Understanding the details of individual cases of deviance is the first step in generalizing into an understanding of failures in systematic processes. A culture of openness is key both to gaining that understanding and to ongoing error prevention.
ncbi.nlm.nih.gov/pubmed/25742063
ncbi.nlm.nih.gov/pubmed/25742063
“Safety clutter”: bureaucratic procedures in the name of reducing risk that don’t, or maybe even increase it. h/t @Dioptre
Scientific grant process aims to avoid funding bad work, but doesn’t, and significantly decreases the amount of good work.
tandfonline.com/doi/abs/10.108…
Scientific grant process aims to avoid funding bad work, but doesn’t, and significantly decreases the amount of good work.
tandfonline.com/doi/abs/10.108…
