Thread by @ShawnMcGuirk

Last workshop of the day, on performance indicators and their relative usefulness, presented by the new prez @CAUT_ACPPU ⬇️ #scipol #science #metrics

There are problems with citation counts!
🔹 Double entries and self citations blur the results
🔹 Everyone cites outrageous articles, and not because they're good
🔹 Citations aren't a good metric for all disciplines
🔹#ECR get less citations

There are problems with impact factors!
🔹 They're derived by formulas, that aren't necessarily known

Problems with h index!
🔹 Disadvantages #ECR
🔹 Bibliometric mistakes can misrepresent
🔹 Only includes citations in certain journals, not from books and monographs

H doesn't even think we should rely so much on H-Index ⬇️

Too much info here to livetweet, but generally:
🔵 Indicators can be useful, but they are too often misunderstood, or are a bad proxy for what is meant to be measured (e.g. p-value debate)
🔵 Bibliometrics lack a view of practical impact (beyond publications)

🔵 Metrics must match the usage. Using outdated or misaligned indicators will lead to misinterpretation.
🔵 Predatory Journals and reproducibility issues aren't well reflected by publication metrics
🔵 Assessment ≠ measurement. Qualitative assessments can be more informative!

🔵 Justifying research dollars is difficult because it's difficult to measure impact of research
🔵 @NSERC_CRSNG Discovery Grants allow 2pg for researchers to describe a narrative of impact, e.g. if a technique developed is now used in more labs around the world, it matters!

🔵 What metrics should count, when evaluating success of university programs? Many are unclear on this.
🔵 CCV does not capture a lot of impactful initiatives that researchers take part in. Focusing on the quality of the work is more important than the quantity

@threadreaderapp #unroll please