Thread by @EikoFried

I'm torn about this decision by @wellcometrust &
@NIMHgov to standardize mental health measures; thread with my thoughts below 🧵

The biggest ✅ is that standardization enables comparison. With 280 scales to measure depression alone, this is overdue.

linkedin.com/pulse/funders-…

However, standardization only helps us compare samples if measures are what we called "measurement invariant", i.e. if they measure the same thing across samples. I am not fully convinced the chosen scales do that.

Which leads me to the biggest ⛔️ of this decision:

Many good reasons to choose these particular scales, such as brevity, but one of the most important reasons from a measurement perspective is validity evidence: do they measure what they claim to measure. And I'm not convinced this is the case. Table below by @JkayFlake 2017.

The PHQ9 for instance, the core measure for depression from now on, measures a subset of symptoms of the (light blue) "QIDS" in the graph below. The QIDS only captures a minority of symptoms featured in the most commonly used 7 scales for depression; the PHQ captures even fewer.

And there is evidence that some of these are really important, such as anger. You can find a lot more information about the challenges of depression measurement is this thread, including many references (e.g. on the importance of measuring anger).

https://twitter.com/EikoFried/status/935098850439847937

@wellcometrust & @mirandarwolpert know as much about this as I do, & decided the ✅ outweigh the ⛔️. I think that's a fair assessment. Here are two things that WELLCOME can do to mitigate the potentially adverse consequences of this decision.

1) Fund research on measurement: let's find out how well these scales do exactly in measuring what they claim to measure. Crucially, this should extend prior work on measurement invariance in these scales to new populations (e.g. ethnicities, languages, etc).

2) Stress the importance of additional measures. Let's not give researchers the justification to write in their measures section: "We used the PHQ-9 because WELLCOME told us to." They should instead have to justify: "We did not use additional measures because ..."

@mirandarwolpert stresses above the difficulty of this decision. This is important because we should not repeat the reification of the DSM, for which many believe today that the criteria are somehow objective or true.

As any historian of psychiatry will tell you: many of the DSM criteria are there for political or historical reasons, rather than empirical reasons. I have an entire talk on this here for depression symptoms in particular (includes video):

osf.io/qdkpw/

Thanks to Miranda & Wellcome for highlighting the importance of consistent measurement in mental health (& kindly linking to our work).

My hope is that this decision will bring more attention to a crucial & under-appreciated topic, but I am worried it will do the opposite.