1/ Signs of scientism vs. legitimate pursuit of truth:
-needlessly complex statistical methods
-emphasis on p-values
-emphasis on demonstrating “effects” vs. understanding mechanisms of action
-focus on hypothesis testing vs developing hypotheses worth testing
-dismissive...
2/ ...of necessary process of observation & inference that could lead to meaningful hypotheses
-deflecting criticism by championing “science” vs defending own methods/conclusions
-spinning weak findings as strong support for preferred conclusions
-arbitrary metrics without anchor
3/ or reference points meaningful to those who are supposed to rely on research
-outcome measures chosen without regard to outcomes desired by those receiving treatment
-use of sham (“intent-to-fail”) control group not designed to control for any meaningful rival hypothesis
4/
-peer review by cronies invested in promoting same conclusions & agenda
-blurring distinction between clinical vs statistical significance
-blurring distinction between research *assumptions* (eg 12 session treatment) vs research findings, discussing both as "evidence based"
5/
-using scientifically-meaningless marketing/PR language in public discourse (“evidence-based,” “gold standard,” “proven”) vs accurately describing actual results
-reporting one-time, short-term outcomes for conditions with known chronic or cyclical course (eg depression)
6/
-reporting “follow up” findings for very brief follow-up interval (eg 6 mos) for conditions with known chronic or cyclical course (eg, depression)
-failing to find significant difference between treatment & control group, then declaring (post hoc) that control treatment is
7/ also an evidence based therapy (in JAMA no less)
-unacknowledged file drawer effects
-willfully ignoring/denying existence of evidence supporting other approaches
I could go on...
Every single one of these things is happening in research on “evidence-based” therapy.
8/ scientism = trappings and methods of science bereft of its spirit and substance
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.