Thread by @DearPandemic

1/ A recent study in the Proceedings of the National Academy of Sciences claimed that mandatory face coverings changed the course of the pandemic:
pnas.org/content/early/…
Sadly, based on all of the available evidence, this study is a HOT MESS. We encourage you to still #maskup 😷

2/ While PNAS is a reputable journal, but the quality of published papers isn't garunteed. We’ve seen recent #COVID19 papers from another top journal, The Lancet, retracted after the data could not be accessed for replication (thelancet.com/journals/lance…).

3/ The REAL peer review happens when the broader scholarly community reads and scrutinizes a paper’s methods and findings. Fortunately, this is happening in real time for #COVID_19 research, especially for papers garnering big headlines.

4/ So what’s up with this PNAS paper?

For an excellent blow-by-blow, we recommend this insightful twitter thread by @NoahHaber:

https://twitter.com/NoahHaber/status/1271518754627678208

5/ One choice quote: “It's hard to write a summary critique of this paper, because it commits ... all of them... You've got ecological fallacy, inappropriate projections, NO treatment of statistical uncertainty (!!!!), attributed causality, non-generalizability, etc.”

6/ Two of the biggest errors, found by our #epidemiologist Nerdy Girls, are:

1) The paper doesn't use a simple epidemiological model of transmission but plots transmission as a linear prediction (a straight line). This isn't a realistic representation of infection spread.

7/
Secondly, the authors (chemists, not social scientists) use case counts in only 3 locations (Wuhan, New York City, and Italy). They attribute deviations from this straight line to only one change in policy (mandatory face coverings).

8/ Why can't this methodology be accurate?

It doesn’t take a statistician to see that 3 locations are not a large sample, and that MANY other things were changing over this time period—like almost EVERYTHING about daily life in all of these places, in fact.

9/ Not to mention in real life we would expect a lag of at least a couple of weeks between any dramatic policy changes and changes in infections.

We could go on. And on. Some have called for the paper to be retracted because it is so misleading.

10/ So...why are we calling attention to it?

COVID-19 research is coming out fast and furious. Mostly, this is a good thing for quickly building a crucial knowledge base.
But like in any line of work, quality and rigor do vary.

11/ In a time when misinformation is truly a matter of life and death, we demand high standards and scrutiny for ALL science.
This is crucial for scientific credibility in the long run. We all want the same thing--CORRECT answers for how best to minimize COVID-19 #risk.

12/ So, in conclusion, wear those masks! But read this research for evidence:
doi.org/10.1038/s41591…
onlinelibrary.wiley.com/doi/full/10.11…