Thread by @WesPegden

Studies like this overstate our quantitative understanding of the benefits of mask mandates.

This does a disservice both by giving the impression that the issue doesn't warrant further investigation, and by drawing attention from the need to improve our response in other ways.1/

https://twitter.com/ScottGottliebMD/status/1319794731505848321

The study undertakes a complicated correlation analysis to attempt to retroactively explain various contributors to epidemic trajectories (from mask usage, population density, smoking prevalence, etc).

Of course, correlation analyses can in many cases produce spurious results.2/

In this case, their results seem strange when subjected to scrutiny.

Before we look at the fits, guess the ranking of the following in importance for COVID:
*) masks
*) fraction of pop in dense areas
*) smoking prevalence
*) fraction of population near sea level

Ok ready?

3/

Their rankings in importance are:
1) masks
2) smoking
3) population near sea level
4) population density.

This can be seen in their SI file here (screenshot below):
static-content.springer.com/esm/art%3A10.1…

4/11

In particular, the coefficient for the importance of population density is 3 orders of magnitude smaller than the coefficient for the importance of smoking prevalence, and 2 orders of magnitude smaller than the coefficient for "near sea level". [*]

5/

In fact, their regression enforced an extra constraint (SI Table 11) that population density have a nonnegative coefficient, perhaps because their regression would otherwise have attributed a negative "causal" relationship between population density and COVID transmission.

6/11

When doing correlation/regression analyses, the whole approach is inherently fraught and it is good to pay attention to red flags.

However, these strange findings about the relative importance of population density are not mentioned in their paper.

7/11

In this particular case, the correlation analysis is used not just to suggest that masks may be useful, but to provide estimates of "number of lives" that could be saved by "universal masking in public".

8/11

Of course, what precisely is meant by universal masking in public is unclear. (Are masks being worn in restaurants while we're eating?)

One of the hard parts about estimating the epidemiological effects of masks is understanding to what extent public mask mandates affect... 9/

mask-wearing in epidemiologically important settings (as opposed to, say, on the sidewalk, outside).

Questionable correlation analyses like this one do not really contribute to our understanding of these difficult questions, meaning that: 10/11

A: Yes it is worth doing more science around masks

B: It is not safe to assume that stricter mask mandates obviate the need for other ideas for how to proceed.

[*] Note for coeffs: not clear if predictors are standardized, but all are fractions bounded between 0 and 1.

11/11