Thread by @dvir_a

How effective are the COVID-19 vaccines? Israel was early to vaccinate its population (~1/3 of pop now with the @pfizer @biontech vaccine) and real-wold data is starting to accumulate. However, as we are learning, calculating efficacy from RWD is complicated. A thread =>

The vaccination campaign in Israel coincided with the beginning of a 3rd wave and a couple weeks later a full lockdown (Israel style). In addition, there are other confounding factors - socio-economic and demographics differences in both the infections and the vaccinations.

This made the real-world data analysis of the effectiveness of the vaccine much more complicated than in the RCT, where all this doesn’t matter because of the randomization. How to tease out all the contradicting factors?

Through the first month of vaccination (until about 10 days ago) there was a lot of frustration because we didn’t see cases, hospitalizations and deaths drop, but actually go up. This made many believe that the vaccines are not effective.

Now, about 6 weeks since the vaccination campaign started there are several studies and evidence that the vaccine is working. However, there is still a debate on how much. The data is mostly in silos, and not shared with the public.

One problem in measuring the effectiveness is regarding case counts- in the RCTs only those with symptoms were tested, in Israel many people get tested without symptoms. The problem is that there is no data on why someone was tested. Below I try to summarise the current evidence:

1. Ministry of Health - shares data of positive cases, hospitalizations, and deaths of vaccinated people, stratified by time from the vaccine. There is no control, and pretty hard to say much about efficacy here.

But we can say that after the second dose 0.07% were positive, which is much less than the general population at that time (about 0.7%). My estimation corrected to days gives 87% efficacy. But again, different ages and socio-economic groups.

https://twitter.com/dvir_a/status/1356503858629984257

2. @segal_eran, @H_Rossman, @ShalitUri - have more granular data. With national data of age of cases and hospitalizations, they show that about two weeks ago trends started changing - cases and hospitalization of those 60y+ started dropping, while 40-60y didn’t.

This uses a kind of natural experiment since in the first few weeks most of those that were vaccines were 60y+. They estimate a drop in cases at around 40% compared to 40-60y.

https://twitter.com/segal_eran/status/1356564979898540032

3. Clalit - the biggest HMO (50%) didn’t share data or preprint yet. A couple of weeks ago they said they see about 33% reduction in cases in those vaccinated.

https://twitter.com/RanBalicer/status/1352572121298436100

4. Maccabi - second-largest HMO (25%) - they compared days 0-12 to days 13-24 after the vaccine and suggest there is 51% reduction. According to the data they present there is no difference up to day 18 and then a major drop in cases.

https://twitter.com/dvir_a/status/1355554956913348610

5. Maccabi 2 - another group using this data showed a reduction in cases and hospitalizations for those vaccinated, starting from day 10 after the first dose. According to this Its not 95% protection, but maybe around ~60% for cases, and ~80% for hospitalizations.

This data is not corrected to any confounding factor, just follows all those vaccinated until Dec 29.

https://twitter.com/KalksteinNir/status/1356450125741256704

In summary, 6 weeks to the started of the vaccination campaign, we are finally seeing the vaccine's effect. The data suggest an efficacy of 50%-90%. How much exactly? going to be pretty hard to know as the data is actually becoming more and more complicated.

My bad, this 41% is a reduction from 2 weeks earlier.

https://twitter.com/shalituri/status/1356620321084833793