Thread by @ReformedTrader

1/ Assessing mandatory stay‐at‐home and business closure effects on the spread of COVID‐19 (Bendavid, Oh, Bhattacharya, Ioannidis)

"After subtracting less-restrictive NPI effects, we find no benefit of more-restrictive NPIs on case growth in any country."
onlinelibrary.wiley.com/doi/full/10.11…

2/ "In the absence of empirical assessment, the effects of NPIs on reduced transmission are assumed rather than assessed.

"The slowing of COVID-19 epidemic growth was similar in many contexts in a way that is more consistent with natural dynamics than policy prescriptions."

3/ "We estimate the unique effects of more-restrictive NPIs (mrNPIs) on case growth rate in the spring of 2020 by comparing effects in other countries to those in Sweden/South Korea.

"The data consists of daily case numbers in subnational administration regions of each country."

4/ "Because the true number of infections is not visible, it is impossible to assess the impact of national policies on transmission.

"We follow other studies and implicitly assume that observed dynamics may represent a consistent shadow of the underlying infection dynamics."

5/ "The average daily growth rate prior to NPIs ranged from 23% (95CI 13% to 34%) in Spain to 47% (95CI 39% to 55%) in the Netherlands.

"The variation of pre-policy growth rates may reflect testing coverage and pre-policy behavior changes."

6/ "While the effects of 3 individual NPIs were positive (contributing paradoxically to case growth) and significant, the effects of about half of individual NPIs were negative and significant.

"The combined effect of all NPIs were negative and significant in 9 of 10 countries."

7/ "After accounting for less-restrictive NPIs (lrNPIs), none of the 16 comparisons (vs. Sweden or South Korea) had more-restrictive NPIs (mrNPIs) that were significantly beneficial.

"The 95% confidence intervals excluded a 30% reduction in daily growth in all 16 comparisons."

8/ "In this framework, there is no evidence 'lockdowns' contributed to bending the curve in England, France, Germany, Iran, Italy, Netherlands, Spain, or the U.S. in early 2020.

"Most of the effect sizes point to an *increase* (not stat. significant) in the case growth rate."

9/ "It is possible that stay-at-home orders facilitate transmission if they increase contact in closed spaces.

"Data on individual behaviors show dramatic decline days to weeks *prior* to implementation of business closures and mandatory stay-at-home orders in our countries."

10/ "School closures cost an estimated 5.5 million life-years for U.S. children during the spring closure alone.

"Considerations of harm should play a role in decisions, especially if an NPI is also ineffective.

"Sweden did not close primary schools in 2020 as of this writing."

11/ "While we find no evidence of benefits for mrNPIs, the underlying data/methods still have important limitations."

* Cross-region comparisons are difficult
* Case counts are a noisy measure of transmission
* The underlying dynamics may be non-linear and contain feedback loops

12/ "We fail to find an additional benefit of stay-at-home orders and business closures above less-restrictive NPIs.

"The data cannot fully exclude the possibility of some benefits. However, even if they exist, they may not match the harms of the more aggressive measures."

13/ The Supporting Information section includes the chart below as well as the code the authors used for their data preparation, analysis, and visualization. (Visit the page below and scroll to the bottom.)

onlinelibrary.wiley.com/doi/10.1111/ec…

14/ Here are the authors' qualifications.

This shouldn't matter much: the paper should stand or fall on its own merits.

Phil Tetlock finds that traditional marks of expertise don't predict forecasting accuracy:

https://twitter.com/ReformedTrader/status/1368252166373220353

Genetic fallacy:
en.wikipedia.org/wiki/Genetic_f…

15/ Related reading:

Effectiveness of three versus six feet of physical distancing for controlling spread of COVID-19 among primary and secondary students and staff

https://twitter.com/ReformedTrader/status/1380586625034383361

16/ Interview with John Ioannidis

* IFR estimation
* Pandemic models may be overfitted & subject to publication bias
* Paper: Assessing mandatory stay‐at‐home & business closure effects on the spread of COVID‐19
* Politicization and ad-hominem attacks

podcasts.apple.com/us/podcast/3-4…