OK here is my take on the 'grievance studies hoax.' I think it shows that generally poorly reasoned, largely unfalsifiable papers with apparently absurd conclusions can get published in top journals in critical-studies-type fields. Fair enough. But it does NOT show a special ...
... kind of 'deep corruption' in such fields as compared to many other fields the hoax authors presumably regard as more 'reason and evidence based' or 'less ideological/political' (e.g., medicine? experimental psychology?). A whole thrust of the "replication crisis" in ...
... psychology, for example, has been that (a) seemingly absurd claims based on (b) insufficiently robust evidence to support such claims can get regularly published in (c) top journals (including Science! and Nature!), insofar as authors 'play the game' of the dominant ...
perspectives/ideologies/dogmas in those fields (see, e.g., "The Rules of the Game Called Psychological Science" journals.sagepub.com/doi/pdf/10.117…). To illustrate, the statistical ritual of Null Hypothesis Significance Testing (library.mpib-berlin.mpg.de/ft/gg/gg_null_…) has effectively been a publication ...
criterion in, not only psychology, but MANY 'more serious' scientific fields for decades, despite the fact that the way it's characteristically been (mindlessly) used has been as a kind of statistical magic that produces loads of nonsense 'findings' that are just noise ...
... and with respect to peer review as an ineffective quality control mechanism, that is absolutely not some special feature of 'critical studies' journals; rather, pre-publication peer review is a deeply broken, deeply unreliable, easily gamed, highly political mechanism ...
in just about EVERY discipline; there have been many studies showing that it is not good at weeding out garbage in, e.g., medicine (see "Classical Peer Review: An Empty Gun" by former BMJ editor Richard Smith …ast-cancer-research.biomedcentral.com/articles/10.11…). In psychology the Stapel Affair showed ...
... that, as long as you play to what's 'sexy' in your field given prevailing orthodoxies, you can *literally make up data* and get published for years in top journals, and only get found out due to whistleblowing (see "Myth of Self Correction in Science ncbi.nlm.nih.gov/pubmed/26168129)
... and in my own research, I analyze how extremely poorly reasoned arguments based on selective citation etc. can end up being official policy of such 'serious' scientific organization as the AAP (see "Cultural Bias in American Medicine" researchgate.net/publication/31…), CDC ...
... (see tandfonline.com/doi/full/10.10…), World Health Organization (see academia.edu/10197867/Betwe…) etc. So, bad reasoning, unfalsifiable claims, statistical rituals, ideologically driven work, politicking, etc. (see "The Unbearable Asymmetry of Bullshit" quillette.com/2016/02/15/the…) ...
... is a MAJOR concern in psychology, neuroscience, economics, medicine, biology, and many other fields, which the 'hoax' authors seem much less concerned about. Now, I am fairly well versed in feminist philosophy, gender studies, etc., and sometimes contribute to those ...
... fields in my own work, and I can attest there is quite a lot of highly political, not-well-argued, empirically under-supported work even in leading journals: you don't have to conduct a hoax, you can just read the literature & see for yourself. But I'm an interdisciplinary...
... researcher and I see a lot of highly political, not-well-argued, empirically under-supported work in a LOT of fields (again, medicine - including 'top' institutions like the CDC - and psychology, are the fields I know best, within which I have sub-areas of expertise) ...
... it's just that you often have to get much more deeply into the weeds in those fields to see the 'corruption' & bullshit because their methods & approaches (& magical rituals, like NHST) are more mainstream & so largely invisible ... Yet I *also* see a lot of extremely ...
... thoughtful work in those fields, by super rigorous researchers, AS WELL AS in fields like feminist philosophy, gender studies, etc., where those latter fields -- when one focuses on the truly excellent work, of which there is plenty -- have the added value of trying to ...
... constantly examine 'obvious' things that are taken for granted, and ask whether they might productively be seen in a different light, and, in doing so, often do add useful tools to our epistemological toolkit for how we should approach various complex topics. ... Indeed, ...
... setting aside 'critical studies' type stuff, even in the most main-stream, hard-core analytic philosophy (with very smart people looking at issues in metaphysics, etc.), it is a *regular thing* to write papers with seemingly 'absurd' conclusions based on 'what everyone knows'
... forcing the thoughtful reader to think deeply about all sorts of things it's easy not to 'see' as deep puzzles otherwise (such as how our beliefs about the external world might be justified, what personal identity consists in, etc. etc.); an outsider to contemporary ...
... metaphysics could easily pull up dozens of papers arguing for 'obviously absurd' conclusions, but this would not entail that those papers were not rigorously argued; it could also reveal the limits of the perspective-taking ability or imaginative capacity of the reader ...
... So it's hard to know what to say in the end. A lot of 'mainstream' science in supposedly rigorous fields is non-reproducible nonsense generated by a combination of statistical rituals, ineffective peer review, publication biases, ideologies and dogmas, politicking etc ...
... and so, the claim that, in 'critical studies' fields it is not *that* hard for a smart person to 'figure out the rules' of publication & get some shady/shoddy stuff accepted in top journals is not surprising, but it also doesn't convince me that there is a SPECIAL problem ...
... with the fields, as such, in these particular regards. That being said, I wonder if there is an interesting asymmetry here. I suspect it actually WOULD be quite hard for a typical gender studies/critical studies researcher to, in a short period of time, master the tools ...
... and learn the rules of publication in many other fields; it seems like, to publish a paper in a top journal in medicine or psychology, you couldn't just whip some stuff up over the course of less than a year and get it through peer review in the most well-respected journals..
... so, the fact that these Sokal-style hoaxes tend to be more or less easy to pull off ON 'critical studies' type journals by smart people from totally unrelated fields, whereas, I'm not aware of cases (and suspect it would be harder) to pull off a hoax in the opposite direction
... may be evidence of asymmetry in something like 'average' epistemological rigor. But there seems a relatively narrow ideological motivation to the authors' hoax in picking out critical studies for condemnation (without charitably engaging w/ what is good in its approaches)
... all of that said, I reiterate, I *do* see a lot of sloppy theorizing that is highly ideologically motivated and not interested in generating falsifiable theories in the types of journals the hoax authors thought to target, and I *do* agree that the cause of social justice ...
... is threatened and undermined when orthodoxies form and you aren't allowed to question them (as I argue here: quillette.com/2016/07/02/in-…); the hoax authors are right that to fight true injustice you need the BEST ideas, theories, data, etc., and that requires getting outside ...
... your bubble where you just talk to other social justice researchers: if the goal is to help the marginalized and oppressed etc., they will NOT be helped in the long run by dogmas protected by blasphemy laws saying you can't critique them. But I think the hoax authors, too,
... could do a better job of approaching those fields/journals in a more charitable way trying to see what is right/good/valuable/productive about them, in the spirit of improving them AND learning from them, rather than the "burn it down" kind of "gotcha" approach they took ...
... Such bomb-throwing tactics to critiquing other fields may, in the long run, turn out to work/be valuable in causing improvement in the general level of rigor/quality (like those 'methodological terrorists' in psychology!); but might also create animus & further divisions ...
... I guess we will see! @NAChristakis predicts that all will come of this is greater effort on part of journals to verify author identities, rather than any kind of soul-searching and improvement. I would like to just see some soul-searching and expanded perspective taking & ...
... charitable/productive engagement happening all around; even when we really disagree w/ someone, there is often something valuable/right in their approach we can learn from (as Michael Hauskeller & I argue here ore.exeter.ac.uk/repository/bit… reviewing Erik Parens on "binocularity").
... Just some initial thoughts & fodder for conversation here ... I am genuinely curious what folks think!
