Thread by @GidMK

So, The Big Mask study has been published, and I thought rather than expound on what the results DID show (everyone's doing that), I might point out a few things that they DIDN'T show 1/n

2/n Study is here, as ever have a read. A very simple, nicely done RCT comparing the advice to wear masks with no such advice in Denmark: acpjournals.org/doi/10.7326/M2…

3/n So, first point. This study says NOTHING about whether mask mandates are good public health policy

Indeed, the authors themselves point this out in the discussion

4/n In fact, we already knew that telling people to wear masks (and providing them with masks) is a relatively ineffective way to stop them from getting a respiratory disease. Universal masking is more complex than that

5/n The study DOESN'T show that masks are ineffective at preventing infection for the person wearing them

I know it sounds weird, but it's true!

6/n See, the authors powered their study (i.e. recruited participants) assuming that masks decreased your risk of infection by 50%, which is quite a lot!

7/n What the study actually shows is that advising someone to wear a mask doesn't reduce their risk of COVID-19 by ~50% or more~, but the results don't exclude smaller benefits

8/n A reduction in risk of infection of 20% would be a bit meaningless to the individual but HUGE at a population level, so this is not a minor point

9/n The study also DOESN'T show that WEARING A MASK is ineffective. It showed that PROVIDING MASKS AND TELLING PEOPLE TO WEAR THEM was ineffective ON TOP OF SOCIAL DISTANCING

10/n As the authors note, compliance was pretty poor. Lots of people were told to wear masks, but didn't

Hard to say what this means for an individual wearing a mask 24/7

11/n Moreover, there was a lot of social distancing already going on in Denmark at the time - this means that we can't really say that MASKS are ineffective but rather than masks didn't reduce infection numbers significantly on top of social distancing

12/n Again, this is not a minor point - masks may indeed not reduce infection numbers much during lockdown, but that doesn't say a lot about their effectiveness at other times

13/n Ok, a technical addition that is nevertheless important. The authors do not report correcting their result for the test sensitivity and specificity of their serology test

14/n Serology tests are used to find antibodies, and they are (as all tests are) imperfect

So, usually we correct for the imperfections to get a better estimate of the true number of people with antibodies

15/n In this case, the study found that 1.8% of people in the mask group had antibodies, compared to 2.1% of people in the non-mask group

But those are just the RAW figures

16/n If we use the Rogen-Gladen estimator, which is a pretty standard correction for test characteristics, we see instead that 1.59% and 1.95% of people in masks/no masks were probably infected, respectively

17/n This sounds like a minor point, but it actually isn't - if only 1.59%/1.95% of people were infected, it means that the study was underpowered for its main analysis, and thus we can't conclude much from the results

18/n Sorry, small correction - I used the final totals of 1.8% and 2.1% not the actual antibody numbers of 1.6% and 1.7% in that calculation. If you apply the correction properly, you get 1.56% masks and 2.09% non-masks

19/n For some context, to find a difference this small, the study would've needed to recruit about 24,000 people, or 12,000 in each group, which is about 4x as big