Thread by @CatieOldenburg

I have a feeling we’ll be seeing a lot of #RegressionDiscontinuity (specifically, interrupted time series) studies evaluating effects of #COVID19 restrictions on all kinds of outcomes. Some thoughts on when RD can be useful and why it might not always be a great idea. 1/n

RD is a method that can be used to evaluate effects of exposures when a continuously measured variable has a cutoff value that can be used to assign treatment (or other exposure) status to those just above or below the threshold. 2/n
ncbi.nlm.nih.gov/pmc/articles/P…

If the cutoff at least partially assigns exposure and is not correlated with outcome, it could be an assignment variable for an RD design. Examples: CD4 counts assigning HIV treatment or calendar time assigning vaccination status based on date of introduction of a vaccine. 3/n

An RCT can be an RD if treatment is assigned based on a random number. If a number is randomly selected from a uniform distribution of 1-10, patients randomized to 1-5 could be assigned treatment and 6-10 placebo. 4/n

Randomization results in exchangeability between groups because randomization causes treatment assignment and is not correlated with the outcome. Groups above/below a cutoff value in an RD can be similar, especially with random measurement error of the assignment variable. 5/n

A classic example is CD4 count determining eligibility for ART in HIV. If patients are immediately eligible for ART when CD4<350 but treatment is delayed >350, the discontinuity is at CD4=350, which is assigning treatment status. 6/n
ncbi.nlm.nih.gov/pubmed/29182641

CD4 counts are measured with random noise. Patients immediately above and below 350 are likely similar to each other in terms of their true disease state. Thus, for a narrow window around 350, receipt of treatment can be thought of as “random” due to random measurement error. 7/n

Using CD4<350 as the assignment variable for treatment, we can estimate causal effects (under some important assumptions) of immediate versus delayed ART on any number of outcomes. A big catch is that these effects are only generalizable to those who are at the cutoff. 8/n

An important assumption is the “continuity assumption”, or exchangeability. With random measurement error, patients close to the threshold may be exchangeable. It is unlikely to hold for those further from the threshold (a pt w/ CD4=50 is not exchangeable w/ a pt w/ CD4=800). 9/n

Because we need to use information from close to the cutoff, there is a bias-variance trade-off in that there are often few patients close to the threshold but using data from those further away can result in loss of exchangeability. 10/n

Using patient data further from the threshold can increase power and we can model the relationship between exposure and outcome to improve validity, but this can still introduce substantial bias (and the functional form can be misspecified). 11/n

There are other important assumptions not discussed here (see linked paper below), but exchangeability is somewhat unique in RD designs and for RDs that are using calendar time, there are specific considerations. 12/n
ncbi.nlm.nih.gov/pubmed/?term=2…

We can also use calendar time and dates of implementation of policies as RD (specifically, an interrupted time series). Time periods before and after introduction of a vaccine can be compared to estimate its effectiveness. 13/n
ncbi.nlm.nih.gov/pubmed/31200889

The continuity (exchangeability) assumption is violated with the use of calendar time as an assignment variable when there are other time trends that affect the outcome. Other interventions happening at the same time as the exposure of interest can result in confounding. 14/n

As a policy implemented at a specific point in time, it’s tempting to think about shelter in place in the context of RD. But it will be important to consider other contemporaneous changes (of which there are many) and the plausibility of exchangeability. 15/n

That’s not to say I don’t think there’s a place for RD/interrupted time series for evaluating the impact of shelter in place and other COVID-related events. But they’ll need to be done thoughtfully with careful consideration of the assumptions and cautious interpretation. 16/n

My biggest concern is that the limitations of these designs won’t be carefully considered, and results will be over-interpreted. I do think they can and sometimes will be done thoughtfully. I’m looking forward to reading, thinking about, and discussing the good ones. 17/End.

ETA: As @NoahHaber describes, I erroneously conflated ITS and RD. This thread is specifically about RD and RDs using calendar time as an assignment variable. I'm excited to see all the interest in RD and will share good ones for COVID that I come across.

https://twitter.com/NoahHaber/status/1262205255090966528