Thread by @shoshievass

1/ Constructing the dashboard to explain our paper (reopenmappingproject.com) involved a lot of careful thinking about what info to display/emphasize and how. The goal of the app was to make the message, methods and results of our paper accessible. Thread👇 for more weeds.

2/ First, data limitations: we build contact matrices from Replica's synthetic population. This is amazing data (e.g. it lets us account for how long ppl spent in the same place) but:

3/ a) it is based on a "typical" day and has poor coverage of rare/big events like concerts; b) it is based on cell pings inside the cities and has poor coverage of travel + of kids; c) it uses Q1 of 2019 as a baseline + modifies based on policies as defined by us.

4/ Second, modeling policies: the main ingredient in our model is the underlying contact graph -- who meets whom and where. Most of our policies work on this graph: e.g. if a workplace is shut down, any contacts that would happen there don't.

5/ We're *not* modeling compliance (e.g. if a workplace is allowed to be open, we assume it will be open) and we're not (for now) accounting for cascading economic effects (e.g. Harvard is closed and so the Starbucks in Harvard Sq has no one to serve and closes).

6/ This ties in crucially to the "masks and social distancing" tab on our dashboard. This one is different from the others because it does not change the existence of contacts, but rather the transmissibility of infections upon contact.

7/ This is a delicate subject, and so we want to be very clear about what we can and can't do here. Under the hood, our model estimates the probability that a sick person will infect someone upon contact w them.

8/ There is no consensus over exactly what the # for this is -- even if there were we'd need to estimate it for our model since our definition of contact is different than what a lab might have, and it would be hard to adjust.

9/ But there is increasing agreement that caution (masks, etc.) have a big impact on this probability.

10/ Our model assumes there is a diff't prob of infection at the beginning of the pandemic, and during the lockdown phase -- and estimates these parameters wrt observed death rates.

Note that our model _also_ accounts for changes in the existence of contacts during lockdown.

11/ The lockdown-time infection probability that we estimate explains virus transmission under shelter-in-place policies where only essential workplaces are open, etc. So this is capturing the effect of caution _on top_ of severely reduced mobility.

12/ We interpret the difference b/w virus transmission estimated before (e.g. first two weeks of March) and during lockdown (e.g. mid March through June) as coming collectively from "caution" -- namely mask usage, social distancing, hand washing etc.

13/ On our dashboard, "rare" corresponds to transmission rates matching the pre-lockdown period; "widespread" corresponds to transmission rates at the height of caution, during lockdown. "Not consistent" and "mostly consistent" correspond to intermediate values.

14/ It's important to note that we can't estimate the effect of mask usage separately form everything else. But our results suggest that non-mobility based behavioral changes happened + had a big impact on R0.

15/ When R0 is low, contacts are less dangerous -- as our dashboard shows.

There is a lot more that we want to do. We're trying to balance adding more nuance, realism and usability to our model, while also adding new locations.

16/ Our focus for this sprint was accessibility -- both wrt the web-app and wrt open source code that anyone can use and adapt. If you want to use our model for policy decisions, we really hope that you reach out ([e-mail]) first.

17/ All models are wrong, and I sincerely hope that our model will be useful -- but in order for it to be useful, it is important to understand what we do and do not know/account for...and what could be known with more localized data.