Thread by @DiseaseEcology

Stanford Serology study preprint just posted that is certain to mislead many people:
medrxiv.org/content/10.110…
It's a serological study, which is fantastic. We need these kinds of studies and data badly. Unfortunately this paper is badly misleading (bordering on purposeful?)

It recruits people via Facebook ads which are clearly not a random sample of the population. The most important subject trait given the influence of age on COVID-19 mortality is: AGE! And yet this study does not present seroprevalence by age or adjust its estimates by age.

It doesn't even present information on the study cohort in enough detail to see the differences in frequency b/w ages of the sample and the pop. They group all19-64 year olds! And there is no measurement of socio-economic status. Both factors (age, $$) likely affect exposure.

There are also questions with the test performance. The manufacturer reported 100% sensitivity for known IgG positives from elisa (75/75) and 92% (78/85) for IgM known +. When they ran the test on samples from Stanford, sensitivity was 68% (25/37) not broken down by IgG/IgM.

Specificity was better (369/371 from manufacturer, 30/30 at Stanford). Poorer sensitivity elevates estimates of True seroprevalence, which leads to their highest estimate: 4.16%.

What should we (safely) conclude from this preprint? I think it's very challenging to interpret the data at all without showing seroprev by age. If seroprev is higher for young people then exposure of old people could be lower.

This matters b/c mortality is ~0 for 0-40 age. Older individs >65 were 3-fold underestimated in survey, so applying their corrected estimates this group would be very problematic.

I've posted comments on the preprint page asking authors to breakdown data by age and adjust for age. If they do, we can get much more from this study. But they didn't even try to collect data on socioeconomic factors and they didn't adjust their results based on prior symptoms.

Perhaps most important, it seems likely that individuals that had COVID-19 symptoms would be more likely to volunteer to be tested and this could lead to substantial overestimate of seroprevalence. Prev symptoms data were collected but not discussed (???).

Conclusion: do NOT interpret this study as an accurate estimate of the fraction of population exposed. Authors have made no efforts to deal with clearly known biases and whole study design (recruiting via FB ad? Really?) is problematic.

Addendum - it would also have been nice to know the professions of sampled individs vs gen. pop. Some jobs likely have much higher exposure. This could also greatly influence results.