,
24 tweets,
6 min read
Read on Twitter
(1) The "Grievance Hoax" (GH) by @peterboghossian et al. was not a "human subjects study" (HSS) subject to IRB regulations. Not in terms of its aims, methods, or results. It could not be replicated, nor could it have been published in a journal for such studies. Thread.
(2) Did the GH not undertake to systematically provide "generalizable knowledge" about the editors and referees for the papers and thus about GS? As a comparison, consider a professor who prank-called members of the Trump administration in order to display its incompetence.
(3) Would this mean the officials were human subjects? Or consider Sacha Baron-Cohen's pranks of members of certain groups as a real life case. If he were a professor, should he have involved the IRB? Clearly absurd. The GH may be more similar to a study, but is not one either.
(4) The GH seems most similar to an audit study, to which it has been compared by foes and friends, even by @HPluckrose. However, audit studies involve simple, repeatable features, control groups and statistical methods that lead to quantifiable results. Neither is true for GH.
(5) For example, audit studies may allow us to say how much more somebody with a typical Anglo-name is likely to get a job interview than somebody with a typical African-American name, given otherwise identical resumes. Such a study could be replicated. The GH could not be.
(6) Even if other people did a similar hoax, it would not be a replication in the relevant sense. And the possibility of replication is essential to studies in the relevant sense, that is, scientific HSSs subject to IRB regulation.
(7) So the GH neither used the methodology of a HSS, nor are its result comparable. Even if it achieves its aims, it neither allows us to say that all GS studies are bogus, nor how many are. It does not and could not provide the generalizable knowledge HSSs aim to provide.
(8) It may still provide insight into GS, perhaps even knowledge. This is what tempts us to say it provides knowledge generalizable in the relevant sense. But this insight (if any) is more akin to that provided by novelists, artists, and comedians such as Sacha Baron-Cohen.
(9) A novelist may create a character and highlight features that exemplify those of certain groups, periods etc.. This can be quite systematic (think Balzac). Parodists and comedian do similar things. The insight can be invaluable, but it is not knowledge of the relevant kind.
(10) The GH created fake papers that also were meant to exemplify certain features of papers in the field they were trying to satirize - highlighting them, while remaining subtle enough to escape detection. (Some argued their papers were too 'normal' to achieve their aim.)
(11) If those papers would be accepted, this would be taken to show there is something rotten in GS, just like the calls of our fictitious professor might show there is something rotten in the state of Trump. This was the point of the GH, but it could not be the point of a HSS.
(12) The primary intent of the GH was to delegitimize & blow the whistle on GS – not the relevant individuals, because to reveal them would be cruel, but a context in which they act. This could not be the primary intent of a HSS, the IRB would be right to prevent such a study.
(13) In what follows, I refer to comments in an article by @jessesingal who, among others, makes the point that because the subjects could be harmed, the IRB investigation of @peterboghossian is proper. I will argue that he rather inadvertently highlights why the GH was not HSS.
(14) The IRB cannot allow HSSs whose primary objective is to harm the subjects or their broader social contexts because obviously that would undermine trust in science and the willingness of the public to participate as subjects. 
(15) But it is obviously perverse to use this to try to shield GS or other fields of the academy against critical scrutiny, which has broadly ethical and political aims, which are essentially different from those of HSSs. Now of course, scientists can also have such motivations.
(16) However, they would have to guard against being unduly influenced by them and such motivations cannot be the primary aim of a HSS. But it can be of a political intervention like the GH, which therefore by its nature also aims for maximal publicity and effect.
(17) While it therefore is correct that the mere acceptance of a paper (as opposed to publication) would be sufficient were this a HSS, it is once again perverse to use this to shield GS against bad publicity, which is the point of GH and harder to avoid in case of publication.
(18) (A further issue is that withdrawing a publication might have endangered the whole enterprise.) And let us also note the extreme irony in the attempt to shield GS against critique by appealing to principles whose purpose is to preserve the non-activist character of HSSs.
(19) Finally, the write-up of the GH was not and could not have been published in a journal dedicated to publishing HSSs. Nor is there any academic discipline that the GH belongs too, while this true for actual HSSs published in the appropriate journals.
(20) While we surely could imagine or find cases which fail to meet one or two of the outlined criteria, I think nothing can reasonably be counted an instance of HSSs which meets none of them. GH is unlike HSS in terms of its methods, its aims, its results and yet other ways.
(21) But this makes urgent the question why not only the foes of the GHs, but also some of its friends and even at least one of its creators have accepted the notion that it involved a HSS. In conclusion I want to discuss two related reasons why this might be so.
(22) Both have to do with a certain kind of scientism. Academics in particular often tend to think that genuine and legitimate intellectual achievements must be scientific. So it is tempting to think that GS can only be delegitimized by something with the authority of science.
(23) The second point has to do with definitions and their role in science. It is tempting to think that the whole meaning of "HSS" can be captured in a pithy sentence or two that define them in terms of a few abstract concepts such as knowledge, generalizable and subject.
(24) But even adequate definitions of this kind only work against the background and in the context of the actual practice of e.g. the academy. When one takes this richer background into account, as I have tried, there can be little doubt that the GH is completely unlike any HSS.
