Thread by @Simon_Whitten

Did the Cass Review disregard the evidence of all but 2 of 103 studies on puberty blockers & hormones to reach it’s conclusions? Yes and no.

I’d like to offer a brief breakdown of what they did & didn’t do & clear-up some misunderstandings.

https://twitter.com/hannahsbee/status/1781582834072703276

Cass commissioned a series of systematic reviews from the University of York, two of these looked at the outcomes of treatment with puberty blockers and with cross sex hormones. We’ll focus on the latter for our example.
adc.bmj.com/content/early/…

The reviews sifted through the available studies and used a variation on the Newcastle-Ottawa Scale (NOS) to grade each study as high, moderate or low certainty (the word used by the authors is “quality”).

NOS works by assessing studies by a list of criteria and assigning points for each box they tick. Based on it’s total point score, each study is designated either ‘low’, ‘moderate’ or ‘high’ certainty.

Certainty is an estimate of how confident the authors are in that the effect reported by the study in question is similar to the true effect (i.e. the effect that would be measured if we had a god’s-eye-view.)

In the CSH review, out of 53 studies the authors assessed 1 as high certainty, 33 as moderate, and 19 as low. The high certainty trial only dealt with side effects. The 19 low certainty studies were excluded from further analysis and the 1+33 were synthesised in the review.

This is what is meant when people like Barnes insist Cass didn’t through out most of the evidence. 34 out of 53 studies made it in. But there’s a twist.

Let’s use psychological/mental health outcomes of CSH as an example.

The review found 5 studies of moderate certainty relevant to psychological health. All 5 supported the conclusion that treatment of trans teens with CSH improved psychological health. There were no findings of worsening psychological health.

What conclusions would you draw from this?

Our intrepid authors draw no conclusions from this whatsoever, instead declaring the evidence for every outcome for which there was no high certainty study “inconclusive.”

Similar is true for other outcomes and for puberty blockers.

This is the sense in which the Cass Review absolutely did ignore almost all evidence on the efficacy & safety of PBs and CSH. The majority of moderate certainty studies were included in the results section but then arbitrarily ignored in the conclusion entirely.

It should also be clarified that the Cass Review didn’t downgrade studies for not being RCTs (the NOS is used to assess non-randomised trials) but did downgrade them for not having a non-treatment control group or if too many members of the control group got fed up and quit.

The really important question that data CAN’T answer is ought we draw our conclusions about the right course of action from the best evidence available while ethically pursuing even stronger evidence?

Or should we favour inaction, no matter how much harm it seems certain to cause, until an arbitrary (and possibly unachievable) threshold is passed?

I’m very much of the view that the former is the only morally defensible course of action.