Thread by @jpogue1

It has been pointed out that my thread yesterday omitted observational data, most notably the Henry Ford study. This was done intentionally, because the observational data for (and against) hydroxychloroquine is honestly confounded beyond salvaging. Allow me to explain.

Those who know me are aware I am highly supportive of well performed retrospective observational studies. It is actually the focus of my research career to date and will continue to be moving forward

However, there are challenges that need to be addressed for the data to have meaning. The primary one is something we call confounding by indication; that is there is a clinical reason that a treating physician chooses an agent (or doesn’t) and it is hard to fix that limitation

The challenge becomes controlling for this. We can do this (sometimes) in ID when there is not a clear answer of which therapy is better; that is, we can identify similar patients who got treatment A and treatment B and compare outcomes in these patients

However, when there are concerns that one regimen is inferior to the other that almost always introduces irreconcilable differences between the study arms because “less sick” or already responding patients tend to be in the inferior treatment arm, making it look better than it is

Interestingly, this is NOT the issue with the hydroxychloroquine data. The issue stems from the fact that at the onset of the pandemic it almost immediately became protocolized at hospitals to either give HCQ to everyone, only the “sicker patients”, or nobody at an institution.

This makes retrospective observational analyses almost impossible because those who do not get treatment are inherently different. Either they are “less sick” and thus have better outcomes for that reason, or they are considered “too sick” and treatment is considered futile.

This manifests in the hydroxychloroquine literature in a number of ways. First, data like those that came out of France initially have no comparison arm (because everyone is treated) – these really are not interpretable as there is no way to understand the impact of the treatment

(link to that example)
sciencedirect.com/science/articl…

Second, you have examples like the data from NYC where 60 – 70% are treated and those who are not are much less ill. Therefore, you see worse outcomes with treatment, but really this is just reflective of disease severity.
jamanetwork.com/journals/jama/…

nejm.org/doi/full/10.10…

Finally, you have examples like Henry Ford where it was standard to treat all patients (literally says in the methods that everyone followed the protocol and it was standard to treat) and 80% of the cohort received HCQ.The question this begs is why weren’t the other 20% treated?

There are three main potential reasons for this: a) control patients died before they could receive HCQ (immortal time bias), b) they were not aggressively treated (made comfort measures only) and/or c) they were treated earlier in the pandemic before the protocol was in place

So, if A and/or B in the previous tweet were true, this would lead to any split in mortality occurring quickly. In the HF study that is exactly what we see. By day 2 you see an 8% difference in mortality and by day 3- 11%. This is the bulk of difference between the treatment arms

Just to put it into perspective, this is what a KM curve looks like when a mortality benefit occurs in a controlled trial where it is randomly decided whether to treat or not. Note the split doesn’t occur until ~day 5 and it is much more gradual (and smaller.)

This hypothesis (that reasons A/B are driving the difference in this study) is also supported by the fact that even though “control” patients were 2x as likely to die they were half as likely to get ICU care or be ventilated (that is, attempts to aggressively treat were not made)

Then there is also the fact that HCQ patients were twice as likely to receive steroids in this study. Many will use this to suggest that steroids are the mortality driver in this study. I actually disagree (although it might be part of the reason!)

I would suggest to you that it is reflective of one of two other things. Either a) control patients were treated earlier in the pandemic or b) it is another sign that control patients were just not aggressively treated (and made comfort measures only). Why do I say that?

In May this institution published an analysis assessing the impact of early steroids on outcomes. Basically, about two weeks into the pandemic they switched to recommending early steroid therapy in patients with worsening oxygen status (which has since proven to be a good move!!)

However, this is an important consideration for assessing this HCQ study where steroid use is much higher in the treatment arm…

The suggestion would be that either they died before they could be treated (with HCQ or steroids), they decided not to treat the patient aggressively (with HCQ or steroids), or that these control patients were managed earlier in the pandemic (before steroids protocol).

If more hydroxychloroquine patients were treated “later in the pandemic” it would be an important confounder in assessing outcomes as in general supportive care has improved over time as we have learned more about the course of the disease

Ultimately, any of these possibilities ends at the same place and this is the issue with this and similar studies. The control group is irreconcilably different than the treatment arm and primed for worse outcomes.

Anyway, the point here is not to dismiss the hard work by many people of sharing their experience. It is more to explain why I think that uniquely in this scenario (HCQ + pandemic) retrospective observational data, whether positive or negative, are just not very informative.

And when you combine that with six RCTs showing a lack of meaningful effect across the spectrum and the pre-clinical concerns, it is enough for me to say it doesn't work

I will always be open to new data. As I have said before, I was very bullish on HCQ prior to our initial clinical experience at Michigan and then the steady stream of data showing no meaningful benefit.