Thread by @PhilWMagness

Let's take a closer look at the @HealthFeedback "fact check" of the JHU study on lockdowns.

This article is going around on social media as a way to discredit the JHU study. But it isn't the impartial analysis that it claims to be.

healthfeedback.org/claimreview/cl…

For starters, @healthfeedback's article relies almost entirely on something called the @SMC_London, which aggregated hostile quotes from pro-lockdown scientists to attack the JHU study.

The @SMC_london is a nonprofit with heavy funding from the Wellcome Trust - the medical financier headed by the UK's lockdowner-in-chief Jeremy Farrar. It regularly publishes hit pieces on research that questions lockdowns, as happened here:

sciencemediacentre.org/expert-reactio…

The @healthfeedback "fact check" leaned heavily on one of @SMC_London's quoted experts, Seth Flaxman of Oxford University.

There's a problem though: Flaxman is not a neutral party. He's the lead author of a competing pro-lockdown study that the JHU paper harshly critiqued.

The JHU study specifically excluded Flaxman's paper because it has deep methodological flaws.

Flaxman et al 2020 use a modeling calibration approach to allegedly test the effectiveness of lockdowns. But their model already assumes that lockdowns work.

The JHU group excluded *all* modeling calibration studies that suffer from similar defects.

It's true that these types of studies are popular in epidemiology journals - but that's a methodological shortcoming of epidemiology itself.

Next the @HealthFeedback "fact check" turns to another non-neutral party: Neil Ferguson, of Imperial College infamy.

Incidentally, Ferguson's lockdown model is also heavily criticized by the JHU study.

The @healthfeedback report quotes Ferguson to accuse the JHU authors of an overly expansive definition of lockdowns. It would appear that Ferguson did not read the JHU paper, as they define "lockdowns" to exclude voluntary measures, but only include mandates.

Note that we can still debate the definition of what counts as a lockdown. But the point here is that the JHU authors were in fact clear about their terms, and were methodical in separating the definitions of various NPIs, contradicting @healthfeedback and Ferguson's charges.

The third "expert" that @HealthFeedback quotes is Samir Bhatt, also listed on the @SMC_London's list of quotations. Who is Samir Bhatt though?

Another non-neutral party. He's a co-author of the same Flaxman paper that the JHU study criticizes for improper methodology.

Note that @healthfeedback does not disclose anywhere that Flaxman, Ferguson, or Bhatt are the co-authors of competing studies, or that their work is specifically criticized by the JHU paper.

Instead it deceptively depicts them as neutral "experts." They are not though.

After finishing with the @SMC_London list, @healthfeedback shifts to another "expert" - one Gideon Meyerowitz-Katz (GMK for short). They depict GMK as an "epidemiologist" and suggest he's a widely accomplished scientist.

He's not though. He's a grad student who tweets a lot.

If we turn to GMK's twitter thread, we quickly find that it isn't the most competent critique of the JHU paper. To the contrary, GMK misreads the JHU paper's reason for excluding modeling calibration studies like Flaxman, and then misrepresents this as a defect of the paper.

Next, @HealthFeedback cites GMK - again on a twitter thread, not a peer reviewed analysis - to claim that the JHU paper reached conclusions at odds with its component studies.

Here GMK misunderstands what meta-analysis does.

Meta-analysis synthesizes and weights multiple studies together to produce a summary estimate of the thing they are examining. As a result, it's not uncommon that the summary estimate will differ - even greatly - from components used to construct it.

It turns out that GMK, aka "Health Nerd," is not the most competent reader either. For example, here he repeats Ferguson's charge that the JHU authors define "literally any intervention" as a lockdown...even though the very next sentence by JHU says the opposite.

To conclude its "fact check," @healthfeedback knocks the JHU study by listing a bunch of other pro-lockdown studies that had opposite conclusions.

Again, that's fine and all but it really amounts to little more than cherry-picking studies that Health Feedback already agrees with

Hundreds of papers have been written on lockdowns, with various conflicting conclusions. Why limit the assessment to only papers that claim the lockdowns worked? And of course @healthfeedback privileges one of those papers above all else: Flaxman 2020.

@HealthFeedback In sum, @healthfeedback:

1. Uses non-neutral "experts" like Flaxman, Ferguson, and Bhatt as its jury on a paper that specifically criticizes the methodology of a pro-lockdown study by...Flaxman, Ferguson, and Bhatt.

2. Misrepresents why the JHU study excluded Flaxman et al

@HealthFeedback 3. Enlists a 4th juror who it depicts as an "epidemiologist." In reality, that juror is a grad student who tweets a bunch of pro-lockdown political talking points about empirical study designs that he plainly does not understand.

@HealthFeedback 4. Caps it all off by cherrypicking a half-dozen or so pro-lockdown studies out of hundreds of papers published on this subject, and declaring that they are correct.

@HealthFeedback In short, @healthfeedback has conducted an irreparably biased "fact check" that privileges the opinions of non-neutral parties to critique a paper that specifically criticizes their own co-authored study. It does not disclose this bias to the reader. Instead, it misleads.