I am in this picture, and honestly it's the most important thing.
https://twitter.com/ass_deans/status/1367824665146761225
Poorly designed quality assessment metrics are arguably even more objectionable than poorly designed primary methods.
Two examples of VERY poorly designed quality metrics are the Newcastle-Ottowa scale and SciScore.
Compare to something like ROBINS-I (which is really good).
Two examples of VERY poorly designed quality metrics are the Newcastle-Ottowa scale and SciScore.
Compare to something like ROBINS-I (which is really good).
One of these days I should really make a quality assessment metric assessment metric (or at least write about what makes the bad ones bad)
I'll give you a start though:
No quality assessment metric can ever determine that a study is high quality or good. It can only ever detect the set of issues it is designed to detect.
"Not found to be poorly designed" does not mean high quality, ever.
No quality assessment metric can ever determine that a study is high quality or good. It can only ever detect the set of issues it is designed to detect.
"Not found to be poorly designed" does not mean high quality, ever.
Additive scores (e.g. indexes) are always misleading if not outright false.
Items should strike at the core of the methods used to make a claim, not at shallow things easier to measure.
Ignoring subjectivity of assessment in favor of "objective" measurement is a peril.
Items should strike at the core of the methods used to make a claim, not at shallow things easier to measure.
Ignoring subjectivity of assessment in favor of "objective" measurement is a peril.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
