Thread by @ukhadds

Over the past decade, various randomized control trials (RCT) have been published that attempt to measure the effectiveness of physical mitigation strategies of airborne disease transmission.

And for the most part, most of them are terrible.

Let me explain why.

A🧵

Airborne disease transmission is extraordinarily complicated, with many steps involved. Consequently, there are many different solutions that have been proposed to limit transmission. They largely involve the removal of exhaled aerosol from the air prior to inhalation.

The most common physical interventions being air filtration, ventilation and masking.

Countless laboratory-based studies have shown how efficient these various techniques are at removing particles from the exhaled and room air.

In short, these techniques are very efficient at removing infectious aerosol from the air.

For filtration and ventilation, modelling can then be used to estimate how efficient the aerosol removal process needs to be for an effect on, say transmission, to be seen.

Ventilation is quantified by the air changes per hour, or the ACH. The more times the air in a room is exchanged with outdoor air, the lower the risk of transmission. The infectious aerosol is being removed from the room and unable to accumulate.

For filtration, a similar value is used. But since the air is not changed but rather filtered, filtration is defined by the equivalent air changes per hour is used, or eACH for short.

To confirm the model/lab-basedestimates of risk based upon ACH, then “real world” studies are then undertaken.

RCT studies are one type of study people used to measure the efficacy of an intervention.

RCT 101:

The structure of a RCT is simple.

You randomly split a population in twogroups, treat one and give the other a placebo.

After a while, the effect is measured.

Fairly simple. This is commonly done to test pharmaceuticals.

In pharmaceutical studies, the two patient groups will be given something like pills where one group gets the pharmaceutical and the other gets a placebo. Each group will be given the same advice on the number of pills to take per day, with/without food, etc.

Important: People have been taught over decades of the risks associated with pharmaceutics (egoverdosing).

Meaning, patients would not be expected to, say, take the entire capsule of pills at once. Thus, it is reasonable to assume that the patients will take the suggested dose.

If desired, blood samples can be taken to confirm that the patients in the treatment group actually do take the pharmaceutical.

If the study takes place in a hospital, researchers will absolutely know the dose being delivered.

Since the dose between the control and treatment groups are well defined and known, any difference in the populations can then be used to determine the efficacy of the drug.

Simple.

Now, what would happen if both the researchers and the treatment group had no idea how much pharmaceutical was delivered?

For example, what if the treatment group were given a mixture of pills that were placebos, doses below the necessary dose and some with the appropriate dose?

What would you expect the data to look like when the treatment is both completely unknown and uncontrolled?

The answer, of course, is that there would be no significance between the control and treatment groups.

The authors would then conclude that the pharmaceutical had no effect, and the drug would be abandoned.

Not good.

This is why it is so important in RCT to have some degree in control of the treatment. To either completely control it, or at the very least KNOW what the treatment is.

For pharmaceuticals, it’s knowing the dose. For air purification, it’s knowing the ACH.

In airborne disease transmission studies, the intervention will be things like filter vs no filter, masks or no mask, etc.

Simple.

Consider this recent study that retroactively tried to assess how well air filtration works at limiting disease transmission.

Let’s look at how the authors describe how they tracked the eACH.

First, they determined how many HEPAs were needed per room. This, of course, depends on the units being (a) operated and (b) at the level they are set.

2 massive assumptions.

The on/off buttons were removed to ensure the filters were left on. Of course, the units could simply be unplugged. To address this, tape was added to “encourage compliance”.

Rather than tape, researchers could measure particle counts in real time, or monitor the power usage for the filter, etc. to estimate eACH or filter usage.

One could argue that this would be more effective than tape, but what do I know.

Here’s a question: What are they not reporting or even trying to measure?

What is missing in the experimental design?

The answer: Measuring the eACH.

AKA: The parameter of interest.

The thing that they are studying. Without knowing this value, the entirety of this study is impossible to evaluate.

To be clear, this entire section is someone explaining in exhausting detail that they do not have any earthly idea what the eACH is but desperately want you to think it’s high.

They objectively have no idea.

If they knew the eACH, they would just report it. But they don't.

Remember, the eACH is the variable whose effect they are measuring, and they have no idea what it is.

This is absurd.

THIS ISN’T A CAVEAT!!!

It’s accurately describing what was not actually measured in the study. This SHOULD NOT be in the “Limitations” section, it should be in the “Results”.

THIS IS A PROBLEM

Again, it is known through lab experiments and model estimates, that below a certain eACH, we wouldn’t expect to see much of an effect on airborne transmission rates.

Studies like this infer that their intervention is high enough without verification and then make sweeping conclusions about the utility of the intervention.

Without measuring the eACH, the study goes from being a hard science/aerosol/engineering study, to one that includes human psychology and behavior

This is fine, but don’t act like it’s assessing filtration, it’s assessing whether people are using the intervention

Very different

Bigger Picture: RCT studies are about quantifying both cause and effect. If you do not accurately measure the cause, then you will never know the cause and effectrelationship.

Imagine reading a pharmaceutical article where they put in the limitations section, “oh, and by the way we have no idea the dose each patient was given”.

Would anyone take that paper seriously? Of course not.

And yet this is common in filtration/ventilation/masking studies

This is general problem in the literature in this area, and it happens repeatedly.

In filtration or ventilation studies, they don’t actively measure the ACH.

In masking studies, they relying on things like questionnaires to estimate usage.

Not good enough.

The larger problem, is that as more of these studies are published and promoted, they create their own momentum. Meaning, they will affect any follow up study.

For example, in the future, the likelihood of a teacher in a treatment group of keeping a portable HEPA filter on in their classroom will be affected by them being told that “they don’t work”. This creates a self-fulfilling cycle.

The fact of the matter is this. RCT studies to measure the effectiveness of physical interventions of airborne disease are not easy to do properly. A lot needs to be considered. And critically, the intervention needs to be quantified. Without that, its utility is limited

Another problem is that this collection of poorly constructed studies are grouped together, and then larger conclusions are drawn. We saw this with the Cochrane mask “study”.

This all is of course made worse when the media (both traditional and social) get their hands on the study.

But that is another subject.

Anyway, those were just some thoughts I had on this issue. Hope you found it interesting.