Looking for badly misleading #COVID19 serology data?
New paper in top journal thelancet.com/journals/lanin…
& new online tool:
serotracker.com/Dashboard
I *hope* it will get better w/ time because it is so misleading now it should be taken down immediately.
Quick thread
New paper in top journal thelancet.com/journals/lanin…
& new online tool:
serotracker.com/Dashboard
I *hope* it will get better w/ time because it is so misleading now it should be taken down immediately.
Quick thread
Serological studies are an extremely valuable tool to understand what fraction of people sampled have been exposed to a pathogen; in this case SARS-COV-2 that causes COVID19.
Huge questions are best answered w/ these studies.
Fatality or IFR ()
Fraction asymptomatic (symptom recall is hard though)
Variation among groups w/ differential access to health care (sciencedirect.com/science/articl…).
In general I'm a huge fan.
Fatality or IFR ()
Fraction asymptomatic (symptom recall is hard though)
Variation among groups w/ differential access to health care (sciencedirect.com/science/articl…).
In general I'm a huge fan.
But doing serological studies in a way that results in interpretable meaningful data is not trivial. This paper/tool seem completely oblivious to this well known fact. If you use inaccurate test, sample skewed set of population, ...
extend your results to a larger geographic area than sampled pop, report your results but no details of how study was done, or any of dozens of other bad practices that we've already seen with COVID19 you will get results that will be badly misleading.
These misleading results can and have been used to push BS stories like COVID19 fatality being similar to the flu; populations being near herd immunity; estimating large rates of case underascertainment, providing poor characterizations of age-specific exposure, etc.
(I've written about this many times:
)
)
I've also written about how fantastic serology studies can teach us tons about the pandemic:
(Note: even this study has deficiencies
)
(Note: even this study has deficiencies
)
So what about this new tool? Does it just take well-done studies from pre-prints or published papers & put them on map in location where study was done? Or at least do a rough cull of studies w/ no details?
Nope! It includes press reports & extends results to whole country!!!!
Nope! It includes press reports & extends results to whole country!!!!
I kid you not! Seroprevalence in all of Italy? 56.31%
All of Austria? 39.74%
All of Russia? 14.67%
Every one of these values is bad misleading joke.
All of Austria? 39.74%
All of Russia? 14.67%
Every one of these values is bad misleading joke.
Where is Italy data coming from?
Didn't Italy release a giant serosurvey today? They did! Did they find seroprevalence was 56%? Nope! 2.5% (see here for details: ).
Data for online tool came from many things including Media reports!
Didn't Italy release a giant serosurvey today? They did! Did they find seroprevalence was 56%? Nope! 2.5% (see here for details: ).
Data for online tool came from many things including Media reports!
"Study" was for 1 part of 1 city. Other data for Italy also. Is it for general pop. of other parts of Italy & has similar value? Nope! Tool has data from serosurveys of Health Care Workers (HCWs) b/c that's representative right? We all work directly with COVID patients right?
Same issues for Russia, Austria, etc.
I cannot believe that this is an online tool & that it was published in Lancet ID.
The idea (compiling seroprevalence estimates globally) is great. The execution is horrendous & very misleading.
I cannot believe that this is an online tool & that it was published in Lancet ID.
The idea (compiling seroprevalence estimates globally) is great. The execution is horrendous & very misleading.
Authors:
Please immediately disable the online Tool.
Then remove all records from the database that are press reports. You clearly cannot verify the details of these "studies". One early press report claimed seroprev in Chelsea Boston was 33%. A later follow up found it <10%.
Please immediately disable the online Tool.
Then remove all records from the database that are press reports. You clearly cannot verify the details of these "studies". One early press report claimed seroprev in Chelsea Boston was 33%. A later follow up found it <10%.
Then stop providing a single value for an entire country (a single value for Russia, really?).
When you have rigorously collected spatially explicit data (which you do for 1 country: Spain), show that on the map by region like this:
When you have rigorously collected spatially explicit data (which you do for 1 country: Spain), show that on the map by region like this:
For all other studies that are on tiny segments of pop (e.g. HCW, blood donors), don't even show the values on the map. Make the user have to click through a set of buttons that says: Do you want non-representative data from HCWs? From blood donors?
Do you want data from people recruited through a Facebook ad by scientists funded by a billionaire who were trying to collect data to support their a priori beliefs that COVID is overblown & share w/ white house? (buzzfeednews.com/article/stepha…)
There are very few medium-to-good quality serosurveys out there & most are from very limited areas (NYC: sciencedirect.com/science/articl…, Geneva: medrxiv.org/content/10.110…). Show these values on the map as the actual tiny little regions studied!
I really cannot believe this is an online tool by reputable scientists that makes it so likely the user will be misled by the data in so many ways.
Project PIs please help!
@RahulAroraAB @TingtingYan_
Project PIs please help!
@RahulAroraAB @TingtingYan_
Update: Authors are improving tool (h/t@Cornejo_gf
). Great! When I went to site now I got warning message. Several misleading country level values were gone (great!). But some are still there - US, Brazil, China.
Need to NOT aggregate estimates by country! Please keep working!
). Great! When I went to site now I got warning message. Several misleading country level values were gone (great!). But some are still there - US, Brazil, China.
Need to NOT aggregate estimates by country! Please keep working!
