Thread by @GidMK

Recently, Professor John Ioannidis, most famous for his meta-science and more recently COVID-19 work, published this article in the European Journal of Clinical Investigation

It included, among other things, a lengthy personal attack on me

Some thoughts 1/n

2/n The article itself is here, and honestly it’s a bit of an odd piece. If I were to commission a review on the small number of SR/MAs on the COVID-19 IFR, I’d probably want it to be written by someone who hadn’t authored one of the 6 onlinelibrary.wiley.com/doi/10.1111/ec…

3/n Moreover, I personally find the entire focus of the piece strange. I do not think it is reasonably possible to accurately estimate the GLOBAL IFR (infection fatality rate/ratio) of COVID-19

4/n The problem with trying to work out a global IFR – i.e. the total number of people dead/infected for COVID-19 across the world – is that both the death AND infection data is scant in most places in the world

5/n For example, this recent systematic review of seroprevalence studies found that even after including more than 400 pieces of research total there was insufficient evidence to infer a truly global estimate thelancet.com/journals/langl…

6/n On the other side of the coin, there’s evidence that in some countries that the death figures from COVID-19 may underestimate the true toll by an order of magnitude (or more!) bmj.com/content/372/bm…

7/n So I don't know if the primary purpose of this paper makes sense

But what is it exactly?

Well, it’s mostly a review of systematic reviews

8/n The author looks at each review and discusses his view on their limitations and successes, then concludes that the best estimate is his own

9/n He spends quite a bit of time on my and @LeaMerone's paper, arguing that we “cherry-picked” evidence to suit our conclusions and that our analysis methods are “overtly implausible”

10/n Now, one thing to note is that these are judgement calls rather than actual scientific critiques. We laid out our methodology quite transparently – saying that this is “implausible” is an opinion, not a fact

11/n That being said, I disagree with many of these statements. For example, this passage argues that we excluded studies in “overtly biased ways” with these three pieces of research

12/n I’m not sure how it is possible to say that something is “overtly biased” when it is transparent and open, but nevertheless there are quite obvious explanations for all of these things (that we give in the paper)

13/n For example, this tabulated estimate includes studies that we reference elsewhere in the review, with 5 of these estimates ~included in our meta-analysis~

It would actually be BAD scientific practice to include these figures twice!

14/n The paper which he co-authored is, I suppose, a matter for discussion – perhaps @LeaMerone and I were presumptuous in reading “selection bias is likely...” as an explicit warning against extrapolating to the entire population of LA County

15/n There are also parts of this paper that are bizarre. It is, for example, not a strength of meta-research to include MORE studies. Indeed, the phrase “garbage in garbage out” is commonly used to describe analyses that do not attempt to exclude poorly-done studies

16/n I would argue that one of the biggest STRENGTHS of our meta-analysis was the time we spent EXCLUDING biased research, because as has now become fairly obvious these studies often overestimate seroprevalence in a population

17/n But overall, I think that Prof Ioannidis' review really shows the issues with having people who have staked their reputation on an issue author perspective pieces on the issue. We all tend to think that our own research is the best

18/n Now, to the personal attacks

I must admit, I was quite shocked to read this published in a scientific paper

I’m not going to go over them, but please do have a read in the paper itself (appendix 1)

19/n For my followers who don’t publish academic research, it’s worth noting that these attacks not only were written by the author, but approved by at least one editor and (usually) 2-3 peers as well

20/n I make no secret of my junior status (it’s there in my twitter bio and every paper I publish), but to say that my research is flawed because of it is a remarkable piece of gatekeeping and I think really quite harmful

21/n It is also worth noting that while I am still doing my PhD, I have been working in public health for more than half a decade, because often the more sought-after qualification is an MPH not PhD

22/n I appreciate the many wonderful people who have come to my defense against these attacks, but in all honesty it’s not me that I’m worried about.

For better or worse, I have a large platform, and I’m not in any huge danger from a professor being publicly mean to me

23/n But imagine, for a second, that I had not been in the news a bit and grown a social media platform. Imagine I was one of 1,000s of faceless PhD students watching a tenured professor at Stanford publicly defame one of their comrades

It’s quite chilling

24/n I may have the wherewithal to defend myself, and I’ll be writing to the journal, but the implication that PhD students have no place in scientific discourse, that their papers are worthless scientifically will, I think, have far greater ramifications

25/n Imagine reading this as a PhD student at Stanford. This is a senior faculty member telling these students that no matter what work they do, their opinions will always come second to professors

Not what I would hope the scientific discourse to be

26/n This issue is not a new one by a long shot. @hertzpodcast covered the issues that PhD students face several times in great detail – I recommend you listen everythinghertz.com/96

27/n I could point out that our paper was reviewed by several very senior epis before we submitted it (including one of the most senior epis in Australia), but that they did not feel they contributed enough to add their names – perhaps this would’ve saved me a tongue-lashing

28/n But the point is that we should not have to have Big Fancy Professors on our paper for it to be considered on its own merits. I’m sure we could have twisted our colleagues’ arms, but we did not think that a professor would stoop to our PhDs as a means of attack

29/n I will be writing to the European Journal of Clinical Investigation. Given that the immediate past Editor In Chief was one professor John Ioannidis, I’m not sure it will do much good, but at least I will have my say

30/n But for anyone reading this who is mentoring PhD students, particularly people at Stanford, I would suggest strongly that you check in and assure them that you do indeed find their opinions and perspectives useful

31/n As to the paper itself? There are obviously more issues – covered here in depth by @AtomsksSanakan – but oddly enough there are also places where Prof Ioannidis and I agree about our paper

https://twitter.com/AtomsksSanakan/status/1375935382139834373?s=20

32/n Perhaps that is because many of the issues he raises about our meta-analysis are pointed out by ourselves in our discussion. Regardless, it would perhaps have been interesting to discuss these in the Journal if not for the unfortunate attacks

33/n Oddly enough, I think that the personal nature of the attack has effectively “silenced” criticism, at least in the EJCI. I do not think I would ever trust the editors of a journal that published attacks such as this and I’m not going to submit an official letter in response

34/n That being said, it’s worth noting that I’m not the only person being targeted here. @LeaMerone, my co-author and a spectacular public health physician, is also being derided for still working on her (I believe) 4th postgraduate degree

35/n More broadly, I think that this sort of punching down is truly unacceptable in academic debate. It is unprofessional, unscientific, and quite depressing to see

36/n Thanks again for every person who has reached out. I cannot express how much it means to me to have the support of such wonderful people as all of you ❤️

37/ One other thing - I do not think it is reasonable to infer from Prof Ioannidis' behaviour to anyone else. I know some of his former colleagues and students and they are all lovely people and excellent researchers