Thread by @ChristoPhraser

Now peer-reviewed in Nature, “The Epidemiological impact of the NHS COVID-19 App”. Here’s a quick rundown of results & what they mean. We estimated that the app prevented several hundred thousand cases from arising. The app works. nature.com/articles/s4158… 1/n

We evaluated the app during the three months after its launch in England & Wales, Oct through December 2020. During this time, the app was actively used by 16.5 million people, 28% of the total population (map a users, map b, cases). 2/n

During that time period, 560,000 app users tested positive for COVID. This resulted in 1.7 million app users being contact-traced by the app. At an individual level, 4.4 contacts were notified per positive user who consented to have their contacts traced (‘sharing keys’). 3/n

In the UK, people who were contact traced were not routinely tested. Instead, we looked at what proportion of people who were contact-traced by the app went on to develop symptoms and later tested positive. 4/n

To make sense of this number, we compare it to manual contact tracing, where people themselves report during a phone interview who they had contact with. These data were summarised very nicely by Lee et al., and I added the app result as a comparator to their plot. 5/n

medrxiv.org/content/10.110…

So in summary, the app has similar accuracy to manual recall of contacts, but is reaching more contacts per person. Of course, manual tracing has broader reach: both approaches have pros and cons, you need both; but it’s useful to see them side by side. 6/n

The next question is what people do with this information. We know that adherence to isolation has been variable for many reasons. Few people adhere perfectly, but many people do drastically reduce their contact rates. We used . covidsocialstudy.org/results 6/n

We used the number of exposure notifications, the secondary attack rate, and the adherence to self-isolation to model the projected number of cases averted from Oct to Dec. This came out as 284,000 (108,000-450,000) cases averted, over a baseline of 1.89 million. 7/n

That was the predicted effect based on what the app was doing. Another way of looking at it was how many cases actually arose in different areas, as a function of the number of app users in that area. Now, correlation doesn’t imply causation, so we needed to be careful. 8/n

To get at the causal effect of the app on the number of cases, we compared only neighbouring local authorities that had a similar number of cases before the app was launched, examples illustrated here. 9/n

We found a very clear signal. In matched neighbouring local authorities, adjusted for poverty, rural/urban score and local GDP having a higher fraction of app users meant fewer cases. Specifically, for every percent point increase in app users, we saw 2.26% fewer cases. 10/n

This matched pretty well with the expectations we had based on earlier simulation work. nature.com/articles/s4174… 12/n

Based on this previous work, we expected very little decrease in infection rates as app use goes up from 0% to 15%. So to estimate cases averted, we extrapolated the national data from actual uptake to what would have happened had uptake been only 15% everywhere. 13/n

Using this approach, we estimated that the app averted 594,000 cases out of 1.89 million cases that actually took place. In other words, contact tracing with the app reduced the second wave of COVID by a quarter. 14/n

Some reflections. First, digital epidemiology will be a key component of future pandemic responses. It has already been in some settings, but this will grow. Getting personalised information in a smartphone may be a transformative ingredient of containing future pandemics. 15/n

Second, with enough foresight and planning, a system can be both privacy preserving and provide very granular epidemiology. People want rich data. Everyone wants to know what they can do that is risky and not risky. They just don’t want that at a cost of mass surveillance. 16/n

Third, this pandemic isn’t over yet, and some places will get new surges of disease before they get vaccinated. Digital contact tracing (Exposure Notification), embedded in operating systems, can be deployed very quickly. The most important thing is how many people use it. 17/n

Fourth, success requires sweating the details to make the whole system convenient and user-centric. As an e.g. a lot of systems are struggling because it’s too hard to get a test result into the app. Covid apps can’t be run in isolation, they are in and for public health. 18/n

Fifth, info is one thing, action is another. The strengths & limitations of other national pandemic responses apply here just as much as anywhere else. A lot of this is about good clear communications, addressing economic precarity, and mitigating distrust. Hard problems. 19/n

Finally. This app has worked. It continues to work. It or its successors will improve. The vaccine, the rapid tests, the app, they all work together to keep case numbers really low. You can be confident that the app is doing its job, and using it will help you do your bit. /end

Massive credit to the team at NHS & DHSC who have built and run this app & programme. Big team effort in this evaluation. Most thanks to everyone who uses the app, it all makes a difference.

@ChrisWymant
@LucaFerrettiEvo
@DaphneTsallis
@OxfordViromics
@mishkendall
@luke_milsom
@cholmesuk
@markbriers

@NHSCOVID19app
@NHSX
@DHSCgovuk
@Google
@Apple
@mr_randers
@matthewsgould

& many others not on twitter