Our NEW systematic review article about diagnosis and prognosis models related to COVID-19 is out now in @bmj_latest
bmj.com/content/369/bm…
📢 This review will be regularly updated in the coming months. Watch this space📢
1/n
bmj.com/content/369/bm…
📢 This review will be regularly updated in the coming months. Watch this space📢
1/n
TL;DR
The evidence base for COVID-19 related diagnosis and prognosis models is weak and reporting quality is generally poor: we can and should do better
We will continue our critical appraisals of new models when they appear in the coming months
2/n
The evidence base for COVID-19 related diagnosis and prognosis models is weak and reporting quality is generally poor: we can and should do better
We will continue our critical appraisals of new models when they appear in the coming months
2/n
What we did
Systematically reviewed and critically appraised articles (including preprints) of COVID-19 related diagnosis and prognosis models developed for individual level predictions
Models to forecast the spread of the COVID-19 infection are not part of this review
3/n
Systematically reviewed and critically appraised articles (including preprints) of COVID-19 related diagnosis and prognosis models developed for individual level predictions
Models to forecast the spread of the COVID-19 infection are not part of this review
3/n
Example
In: model that predicts length of hospital stay of a COVID-19 confirmed case based on their age, history and lab results
Out: model that predicts number of COVID-19 confirmed cases on April 10 in the Netherlands
4/n
In: model that predicts length of hospital stay of a COVID-19 confirmed case based on their age, history and lab results
Out: model that predicts number of COVID-19 confirmed cases on April 10 in the Netherlands
4/n
Why we did it
COVID-19 is putting a strain on healthcare systems. Diagnosis and prognosis models aim to help in obtaining earlier and more accurate diagnosis and evidence based estimates of prognosis in patients with COVID-19, considering multiple factors that can affect it
5/n
COVID-19 is putting a strain on healthcare systems. Diagnosis and prognosis models aim to help in obtaining earlier and more accurate diagnosis and evidence based estimates of prognosis in patients with COVID-19, considering multiple factors that can affect it
5/n
Our aim
Differentiate between the diagnosis and prognosis models that could be beneficial for patient care and the poorly developed or reported ones that may do more harm than good when applied in clinical practice
6/n
Differentiate between the diagnosis and prognosis models that could be beneficial for patient care and the poorly developed or reported ones that may do more harm than good when applied in clinical practice
6/n
What we found
31 models, ranging from models that aim to quantify the risk of infection in the general population to machine learning algorithms on chest CT scans that assist in determining diagnosis
7/n
31 models, ranging from models that aim to quantify the risk of infection in the general population to machine learning algorithms on chest CT scans that assist in determining diagnosis
7/n
The bad news
All models were appraised at high risk of bias using a standardized tool for prediction models called PROBAST: development.probast.org
Reported estimates of model performance were generally too optimistic and reporting quality was often unacceptably poor
8/n
All models were appraised at high risk of bias using a standardized tool for prediction models called PROBAST: development.probast.org
Reported estimates of model performance were generally too optimistic and reporting quality was often unacceptably poor
8/n
Reporting quality can be improved
Even for more experienced model developers, we highly recommend following the TRIPOD guidelines to improve reporting, especially now that model development is often done under high time pressure: tripod-statement.org
9/n
Even for more experienced model developers, we highly recommend following the TRIPOD guidelines to improve reporting, especially now that model development is often done under high time pressure: tripod-statement.org
9/n
The good news
We point to a few models that, despite their limitations, may be worth considering for future validation studies
Some of the predictors were identified consistently across studies can also serve as a useful starting point for new model development
10/n
We point to a few models that, despite their limitations, may be worth considering for future validation studies
Some of the predictors were identified consistently across studies can also serve as a useful starting point for new model development
10/n
Thanks
To all co-authors involved and to @laure_wynants especially as a co-lead
And to @bmj_latest for a quick editorial process and allowing us to periodically update this review over the coming months
11/n
To all co-authors involved and to @laure_wynants especially as a co-lead
And to @bmj_latest for a quick editorial process and allowing us to periodically update this review over the coming months
11/n
Final shoutout to @ISPM_ZOAP @HiraImeri @nicolamlow for their living evidence database that made our literature searching a lot easier: ispmbern.github.io/covid-19/livin…
12/n
12/n
