Thread by @JonathanShedler

Plain English summary: people who don’t know how to talk about painful emotions are more likely to act on them.

1/ Signs of scientism ⚠️ vs. legitimate pursuit of truth:

-needlessly complex statistical methods
-emphasis on p-values
-emphasis on demonstrating “effects” vs. understanding mechanisms of action
-focus on hypothesis testing vs developing hypotheses worth testing
-dismissive...

2/ of necessary process of observation & inference that could lead to meaningful hypotheses
-deflecting criticism by championing “science” vs defending own methods/ conclusions
-spinning weak findings as strong support for preferred conclusions
-arbitrary metrics without anchors

3/ or reference points meaningful to those who are supposed to rely on research
-outcome measures chosen without regard to outcomes desired by those receiving treatment
-use of sham (“intent-to-fail”) control group not designed to control for any meaningful rival hypothesis

4/
-peer review by cronies invested in promoting same conclusions/agenda
-blurring distinction between clinical vs statistical significance
-blurring distinction between research *assumptions* (eg duration of treatment) vs research findings, referring to both as evidence based

5/
-using scientifically-meaningless marketing/PR language in public discourse (“evidence-based,” “gold standard,” “proven”) vs accurately describing actual results
-reporting one-time, short-term outcomes for conditions with known chronic or cyclical course (eg depression)

6/
-reporting “follow up” findings for very brief follow-up interval (eg 6 mos) for conditions with known chronic or cyclical course (eg, depression)
-failing to find signif difference between treatment & control group, then declaring (post hoc) that control treatment is also an

7/ evidence based therapy (in JAMA no less)
-unacknowledged file drawer effects
-willfully ignoring/denying existence of scientific evidence supporting other approaches

I could go on.

Every single one of these things is happening in research on “evidence-based” therapy.