,
17 tweets,
8 min read
Read on Twitter
Are there individual differences in response to antipsychotic treatment?
Excited to share our new paper, now online @JAMAPsych. Here's an overview of what we did and what we found.
Excited to share our new paper, now online @JAMAPsych. Here's an overview of what we did and what we found.
So what's the problem here? Isn't it obvious that patients respond very differently to treatment? After all, that's why everyone talks about personalizing medicine, right? Well, not so fast. It turns out that things are actually less clear: nature.com/articles/d4158…
Statisticians such as @stephensenn @statsepi @f2harrell and others have repeatedly pointed out that the typical responder analysis of RCT data is quite problematic: darrendahly.github.io/post/2017-08-1… 
What is it that makes responder analysis problematic? When we look at outcome data from an RCT, isn't it evident that some improved more than others? That's what the data shows, no? Why can't we just define a threshold and use it to label patients as responders & nonresponders?
Well, we certainly can. But here's the thing with labels: they stick. Once we label a patient as a responder we might think that we have established response as a permanent feature of that patient. The problem is that we haven't. We haven't seen that patient under placebo.
Also, the response that we observe in an individual in an RCT is not just the response to treatment. It's a confound of treatment effect, placebo effect, regression to the mean, and measurement error. And we all know that. It's the reason why we need a control group in an RCT.
What's maybe less obvious is that this means that differences observed at outcome in a group of treated individuals may still be compatible with a constant treatment effect across individuals. We simply don't know. We'd have to see the same individuals under placebo.
As a side note: this also means that analyses aimed at classifying responders and nonresponders with #MachineLearning will have to be interpreted with caution as long as the labels are derived from RCT outcome data.
Perhaps more surprising: Not even a placebo condition is enough. As @stephensenn nicely showed: when we see 30% response that could mean 30% of the patients respond 100% of the time or 100% of the patients respond 30% of the time: bmj.com/content/329/74…
Yes, to be confident about individual differences in response we have to work hard. We'd have to see the same patients repeatedly under treatment and under control. Is this feasible? Possibly not. But without it we don't really know. So what can we do?
We might ask how the variances of treatment & control should look like if there were individual differences in response. An excellent recent paper did just that: f1000research.com/articles/7-30/…. Briefly, we would expect the variance in the treatment group to be larger compared to control
Which would indicate that either all people respond differently (a case for personalized medicine) or that a subgroup responds differently (a case for stratified medicine). That's a testable hypothesis: We'd expect the treatment variance to be larger than the control variance.
In the second part of our paper we tested this hypothesis and asked: is the overall variance of treatment larger compared to control when we summarize across antipsychotic drug trials of the last decades?
The answer is that it isn't. Perhaps surprisingly, we found that the variability of treatment is actually slightly smaller compared to control (3% smaller, with a 95% CI of 5% - 1%). What could this mean?
Are we saying that we have ruled out differences in treatment response with this analysis? Certainly not. In fact, the result is compatible with scenarios where a subgroup of patients does while another doesn't benefit from treatment (figure from the paper by Cortes et al.)
Importantly though: As the previous figure nicely illustrates, a result as the one we found could mean that the subgroup that responds better would also have to be more symptomatic at baseline. That is something we should test, in treatment resistant psychosis etc.
Thank you to @SteWinkelbeiner for your amazing work on this project & for sharing data and R code @OSFramework & to Stefan Leucht for sharing the data with us & to @wviechtb for the excellent #metafor package & and to our reviewers for helping us making this a better paper!
