Thread by @tylerblack32

1/ Thread (and it's a big'un!):

Issues with Psychotherapy Research

It is 2021 and virtually none of following major issues have been largely addressed, which severely undermines the practical value of the research.

2/ As a psychiatrist, I am trained in both pharmacotherapy and psychotherapy. I care deeply about evidence in both areas and am a strong proponent of "as many useful tools as I can have to help my patients."

3/ I support, <3, and provide psychotherapy. This is NOT a debunking of it, nor would I dissuade a single soul from taking it or providing it. For children and adolescents, psychotherapy is essential and should be considered first line in most situations.

4/ However, psychology research needs considerable upgrading. Let's review. I've only selected this GIF because of the cartoon of knuckles cracking. I promise, I have the kindest of expressions right now...

5/ ISSUE 1:
Wait list / "Treatment as usual" controls.

It has been demonstrated over and over, that if you choose to compare your psychotherapy to either of these control groups, you will amplify the apparent effectiveness of this therapy.

6/ ncbi.nlm.nih.gov/pmc/articles/P…

Zhu et al showed that placebos outperform waitlists, and provide benefit, such that the ES of CBT ("1.2") is HALVED by comparing it to psychological placebo, & even in those studies there is major risk of bias.

7/ Cuijpers et al found when adjusting for waitlist, the effect size of ALL psychotherapies for adult depression dropped by 19% for simply removing adjusting for wait-list controlled studies. (more on this paper later)

ncbi.nlm.nih.gov/pmc/articles/P…

8/ What are we talking about here?

Let's study a therapy: "Purple hat therapy." It's a new form of therapy, except we wear a purple hat!

So we'll design the study comparing people on a waitlist to people doing PHT.

Wow! Effect size 1.8?!?!

Our results are great!
#PHTWORKS

9/ Of course, you and I both know that wearing a purple hat likely does not influence therapy much, aside from patients who very much prefer purple hats.

Let's compare it to therapy WITHOUT purple hats!

Uhoh! Effect size 0.13 (negligible)
Maybe #PHTDOESNTDOMUCH

10/ Waitlist controls absolutely DOMINATE psychotherapeutic Therapeutic Trials, along with the nebulously coined "Treatment as usual". This is basic interventional science: you need to control for as much as you can, this is why ACTIVE PLACEBOS are SO IMPORTANT in med studies.

11/ At this point, it has to almost be assumed as intentional. It has been well documented in many many many studies that head to head with other treatments, most psychotherapies perform similarly. That is, most of what makes therapy work is NOT what is unique about the therapy.

12/ Issue 2: Not acknowledging Side Effects

Imagine a drug study showing that my new patented drug Tylernol (intentional! Sorry!) worked for depression. And my whole research report contained ZERO mention of side effects.

How would you feel?

Welcome to psychotherapy research!

13/ The side effects of psychotherapy are SIGNIFICANT.
Moritz et al sent questionnaires to 135 people receiving various psychotherapies for depression.

52.6% reported at least one adverse event.

link.springer.com/article/10.100…

14/ These events comprised of malpractice (27%), unwanted side effects (38.5%), and unethical conduct (8.1%).

15/ In my realm, kids and suicide, one study showed:

88 kids
MED-FREE therapy for depression
8 (9%) new suicidality in the first 3 weeks
11 (12.5%) within 12 weeks

ajp.psychiatryonline.org/doi/full/10.11…

16/ It is SLOWLY getting better yet still in the year of a deity 2021, it's about 16:1 that a medication trial mentions side effects vs. a psychotherapy trial.

17/ ISSUE 3: RESEARCHER BIAS

Also known as "allegiance" bias, this is the bias of the researcher themselves having special belief or skill at the thing they are studying, so the results are not because of the thing but because of this bias.

18/ Everyone can quickly spot this one:

If PharmaCorp believes in, funds, develops, and markets a drug called Pharmatolol, and funds, publishes a study showing it works, what's the first thing a critic would say?

CONFLICT OF INTEREST! PharmaCorp really wants this to work.

19/ Dragioti et al found this. Though there is high risk of bias in the studies that look at it (irony), the factors:
* developing the therapy
* researcher trained the therapists
* all forms EXCEPT CBT showed this

link.springer.com/article/10.118…

20/ And yet, embarrassingly, in psychotherapy trials, the same team found that 3.2% of researchers declared their allegiance. THREE POINT TWO. Imagine if 3.2% of med trial authors declared their conflicts of interest.

bmjopen.bmj.com/content/5/6/e0…

21/ There are many other sources of conflict of interest and bias, and in medication trials there are systematic ways to look for them.

Back to the Cuijpers article, when ALL controls and bias was adjusted, the effect size of psychotherapies?

DOWN 51%

22/ When we control for the placebo measure and the risks of bias, the effect size for psychotherapies ranges from 0.2 (negligibly small) to 0.35 (mild).

23/ Re: Publication bias, Cristea et al showed the effect size of psychotherapies for borderline personality disorder fell from a small g=0.35 to a TEENY g=0.2 (0.2 is the "sig cutoff" bleh)

jamanetwork.com/journals/jamap…

24/ I truly believe the next generation of psychological researchers are going to acknowledge this reckoning, and I have faith that the scientific process will bear its fruit. However, without addressing major issues, psychological treatments are constantly oversold.

25/ The reckoning occurred in pharmacotherapy, and even then we are still dealing with the ramifications of it, but at least it's spotted and identified in most research. Psychotherapy needs to play catchup here.

26/ Notes:

1) there are MANY other issues with research in this field, from the selection of scales vs patient-centered questions/meanings, replication, representative diversity in patient groups, and lack of economic testing, just to name a few.

THIS IS NOT EXHAUSTIVE.

27/ 2) this is NOT meds vs psychology.

As a @Royal_College Psychiatry and Child Psychiatry trainee, I am both trained in pharmacotherapy and psychotherapy, and I firmly believe evidence supports the use of both for our patients.

28/ What this is is a calling out of the inflation of psychotherapy's "evidence" due to poor research design and bias.

29/ 3) I have ZERO interest in psychology vs. psychiatry. I love my psychology colleagues and work alongside them as colleagues with different expertises and rely on psychological expertise HEAVILY. I absolutely respect and consult psychologists regularly.

30/ 4) Similarly, I don't care about brand x vs brand y of therapy. Y'all look like this to me.

31/ I firmly believe that:
* the relationship between the patient and the therapist, and a host of NONSPECIFIC factors, make up the majority of psychotherapy value, but it has huge value!
* the guild wars within brand of therapy do not put the patient first and are embarrassing

32/ Sorry? for the length. As always, I try to do my best to compress single ideas to single tweets, which leads to communication condensing. There may be unintentional errors do to this.

Conclusion:
I love science and I desperately want the science of psychotherapy to be better