Thread by @JasonSalemi

#COVID19 #vaccination data.

Vax estimates above 100%.

Millions of people seemingly failed to complete their initial 2-dose series.

Unexpectedly low rates of booster uptake.

A thread explaining how these data might be D-I-R-T-Y.

1/16

bloomberg.com/news/articles/…

PROBLEM #1: WRONG PLACE OF RESIDENCE

People may be assigned to a county based on where they got the shot and not where they lived. [that's the numerator]

Since the denominator is based on place of residence, the % vaccinated can end up out of whack.

2/16

Problem occurs when misclassification in one direction (calling a non-Hillsborough resident a Hillsborough resident to a greater extent than the opposite direction (calling a Hillsborough resident a non-Hillsborough resident).

More likely to happen for specific age groups.

3/16

PROBLEM #2: BAD POPULATION ESTIMATES

Even if the number of people vaccinated (numerator) were perfect (it's not)...

Not knowing how many people actually live in a particular place (bad denominator) can result in a very unrealistic % vaccinated.

4/16

In the hypothetical example below, the population estimate for Hillsborough county people 65 years or older was about 258,000 to low.

Underestimated denominator = higher reported % vaccinated.

5/16

PROBLEM #3: FAILURE TO LINK SHOTS TO THE CORRECT PERSON

I don't assume to know how every jurisdiction links together all shots administered to the correct people, whether at CVS, a dr office, church, drive-up site, community event

But I imagine the process is imperfect.

6/16

Example 1- person #5 in the fig below got two shots of Pfizer in October.

She's 'optimally immunized'.

But, her second dose was accidentally linked/assigned to someone else who was never actually vaccinated.

Now, both are erroneously classified as 'partially immunized'.

7/16

Example 2- person #5 in the fig below got two shots of Pfizer in May, boosted in Nov.

She's 'optimally immunized'.

But, her booster dose was accidentally linked/assigned to someone else who was never actually vaccinated.

(cont'd on next tweet)

8/16

Now, she (optimally immunized after boosting) is erroneously classified as 'immunized with waning immunity'.

The other person (never vaccinated before) is erroneously classified as 'partially immunized'.

9/16

Example 3- person #4 in the fig below got two shots of Pfizer in May but never boosted.

He's 'immunized with waning immunity'.

But, his second shot was accidentally linked/assigned to someone else who was never actually vaccinated.

(cont'd on next tweet)

10/16

Now, he (waning immunity) is erroneously classified as 'partially immunized'.

The other person (never vaccinated before) is also erroneously classified as 'partially immunized'.

11/16

From what I can tell, the result of PROBLEM #3 (linkage problem) tends to be:

- we underestimate the # of people optimally immunized
- we underestimate the # of people not immunized
- we overestimate people with "just 1 dose" (partially immunized)

12/16

I know I haven't discussed vaccine tourists & snowbirds, but these have been discussed a lot.

I also know there are plenty of other data issues, but these seem to be some common ones.

Keep these in mind while interpreting the data.

13/16

If you're wondering about the classification of vaccination status that I've been using, please check out our pre-print (paper under peer review now).

medrxiv.org/content/10.110…

14/16

Could never get the word out without my high-profile peeps!

@bethlinas @nataliexdean @angie_rasmussen @berthahidalgo @EpiEllie
@BillHanage @bhrenton @EricTopol @JenniferNuzzo @meganranney
@aetiology @ashtroid22 @ashishkjha @Cleavon_MD @DrLeanaWen

15/16

Could never get the word out without my high-profile peeps!

@Theresa_Chapple @uche_blackstock @WesElyMD @WhitneyEpi @yoreekoh @EllieMargolis @mugecevik @kaikape @jtrebach @profshanecrotty @choo_ek @paimadhu @Surgeon_General

16/16

Great points made by @EpiMegan.

In my examples, you can also consider incorrect assignment to the wrong person as incorrect assignment to a duplicate record for that person, which results in similar aggregate classification errors...

Thanks Megan!

BONUS THREAD

https://twitter.com/JasonSalemi/status/1473291669298573316?t=XsWrShV8lhRXH-CgjkBLXA&s=19