Thread by @DivaJain2

Surprisingly many who were demanding triple peer reviewed data from Bharat Biotech on Covaxin’s effectiveness have enthusiastically embraced WHO numbers without a murmur on methodology and data quality.

A long semi technical thread that will beg your patience and attention 1/n

Notwithstanding the opacity with which WHO has spliced data from different sources to create a single subnational panel for India, there are deeper methodological concerns about these estimates. 2/n

WHO acknowledges that modeling excess deaths for India is “most complex” –

From the report -

“We consider the most complex subnational scenario in which the number of regions with monthly data varies by month, using India as an example.” 3/n

Since monthly mortality data is not available at a country level, WHO uses state wise data.

The problem is that state data is also not available for all months and the number of states for which data is available changes by month. 4/n

So what WHO does is try to project national deaths based on limited state level data. In order to do so, it relies on a paper by Karlinsky (2022) who uses excess deaths in the Cordoba province to estimate excess deaths for Argentina.

But herein lies the rub. 5/n

For Karlinski’s method to yield unbiased estimates, the proportionality principle must be satisfied.

Meaning the share of COVID deaths of the State/region (being used for national projections) out of total national deaths should be stable throughout the projection period.
6/n

In other words, for example, if Kerala accounted for 30% of all COVID deaths in June 2020, it should account for the same in February 2021 as well!!

7/n

Karlinski is crystal clear about this -
“This projection (state to national) makes use of the stability of the spatial distribution of deaths within countries and is appropriate where the spread and toll of COVID19 are similar between the subnational and national data used”
8/n

Karlinski even shows evidence that this spatial distribution is very similar between Cordoba and rest of Argentina in Figure 1, panel B of his paper –

see for yourself - iris.paho.org/bitstream/hand…

9/n

WHO also acknowledges this –

“If, over all regions, there are significant changes in the proportions of deaths in the regions as compared to the national total, or changes in the populations within the regions over time, then the approach will be imprecise.”

10/n

BUT UNLIKE Karlinsky, WHO provides NO EVIDENCE to support that this holds for the subnational/statewise data used for India!!!!!

11/n

Eyeballing data from India, I can see that different states went through peaks and troughs at different points in time during the pandemic. All of them did not correspond to the national peak/trough.

12/n

WHO should have validated this and perhaps used data only from those states that mirrored the national distribution of COVID. By not doing so, WHO’s methodology seemingly violates the most basic assumption upon which it is based.

13/n

I understand that lacking good data, estimating excess deaths is difficult.

But a good (and honest) statistician always validates (or tries to) her assumptions and acknowledges the limitations of her estimates.

Something WHO neither addresses nor acknowledges.

END.