Thread by @KertViele

(1/n) I saw this recently arguing against response adaptive randomization (RAR) in platform trials. There are good counterarguments to the objections…here is a tweetorial on the RAR debate (refs at end) and why RAR is used in many modern platform trials.
ncbi.nlm.nih.gov/pubmed/32222766

(2/n) RAR is an adaptive experimental method that employs multiple interim analyses. At each interim allocation for poorly performing arms is reduced (perhaps to 0) and allocation for better performing arms is increased.

(3/n) RAR was initially proposed to benefit patients within a trial (“internal patients”). The intent is that as the trial evolves, more patients are placed on better performing arms, resulting in better outcomes.

(4/n) While RAR benefits internal patients, trials are also conducted to benefit society. We need trials to select the correct arm for future, “external”, patients. RAR can be helpful or harmful in this regard, and many papers talk past each other addressing different settings.

(5/n) The key splits in the literature are determined by the number of arms, either 2 arm (control vs single treatment) or multiple arm (control vs multiple treatments). In multiple arm settings we also need to consider how control is allocated. Start with two arms….

(6/n) In two arm trials, you have to alter control and treatment allocation together (there is nothing else to change). As noted by Korn Freidlin 2011, RAR loses power, since 1:1 allocation is optimal. Thus benefit to internal patients can come at a cost to external patients.

(7/n) Additionally, two arm RAR doesn’t handle time trends well (sidebar, unclear how big these really are). The basic idea…suppose in the first half of a trial control cures 20% and treatment cures 30% and the second half has control 30% and treatment 40%.

(8/n) Suppose you move away from control in the second half of the trial. First half allocation is 50-50 ctrl:trmt, but the second half is 10:90 ctrl:trmt. Your control averages 21.7% and your treatment 36.4%, and you overstate the treatment effect.

(9/n) Since you have randomized away from control, it’s difficult to estimate time effects separate from treatment. While the power is a bigger deal to me, combined I’m not fond of RAR in two arm settings.

(10/n) There are other objections for two arm settings. For example Thall et al 2015 note biases and increased variability in point estimates. These objections seem particular to their hyper aggressive form of RAR, rather than an inherent property of all RAR.

(11/n) For example, in their paper in one setting they note biases of 9% and a 25% of overstating benefit by twofold. By changing the interim schedule to something more common, I achieved a bias of -0.17%, essentially nothing, and a 0% chance of overstating benefit by twofold.

(12/n) The final objection in Thall et al 2015 is that while two arm RAR benefits internal patients on average, there is a chance of imbalance in favor of the weaker arm. This is true, but I don’t understand. If something is better 93% of the time and worse 7%, why is that bad?

(13/n) Ok, so two arm RAR doesn’t sound great to me, so why I am spending time on this tweetorial? Because multiple arm and platform settings are much much different, if you handle control allocation correctly.

(14/n) Many “anti RAR” papers argue RAR performs poorly in the multiple arm settings. Some make this argument intuitively while others like Wathen and Thall 2017 have examples. These examples involve reducing control over time.

(15/n) Current RAR designs simply don’t do this. If you want high power comparing the best of multiple arms to control, you will need data on both that best arm and control. Thus, modern RAR trials maintain control allocation, using RAR only among the active arms.

(16/n) Example, suppose N=200, 3 active arms and control. With equal allocation, the best arm to control comparison has 50 patients each. If you reduce control allocation, you may end up with 15 control, 120 on the best arm. This is worse than 50 each because of limited control.

(17/n) But if you maintain control, and perform RAR only among the active arms, you can achieve 50 controls versus ~100 on best arm. That is better than 50:50. You can even allocate more to control and end up with something like 65 controls, 100 best arm.

(18/n) These RAR variants, maintaining control, are superior to equal randomization with properties equal to other adaptive alterrnatives. See work by @JMSWason @JasonConnorPhD @xelamb @LorenzoTrippa @RogerJLewis @statberry and others. (refs below)

(19/n) I and colleagues (Anna McGlothlin, Kristine Broglio, and @statsninja) explored control allocation in multiple arm RAR trials in a recent Clinical Trials paper. We explored power, estimation, arm selection, and treatment of internal patients.
ncbi.nlm.nih.gov/pubmed/31630567

(20/n) We considered 8 designs. Three designs had fixed allocation, with 25%, 40%, or 50% allocated to control and the remaining arms allocated equally (names F25, F40, F50). A fourth design employed RAR allowing diminishing allocation to control (RAdjCtrl)

(21/n) 3 more designs employed RAR only on the active arms, with fixed allocation to the control arm (R25, R40, R50). A final design allowed for increased allocation to the control arm to match the highest active arm (RMatch)

(22/n) The RAR designs that maintain control dominated fixed allocation on every metric (again, for comparison to fancier adaptive designs, see the references below). The figure below shows power. R25, R40, R50, RMatch perform well…RAdjCtrl performs poorly.

(23/n) Mean square error is below (remember low is good). Again the RAR designs the maintain control perform well. Again RAdjCtrl performs poorly.

(24/n) Finally, in a multiple arm trial we want to have the correctly select the best arm. As the table below shows, the RAR designs maintaining control allocation perform better than equal allocation designs at arm selection.

(25/n) We did not discuss temporal trends in our paper, but with control allocation maintained you can simply block over time and adjust for time effects. Some modern platform trials like ISPY2 employ a model for the control arm over time, called a time machine.

(26/n) In summary, if you need comparisons to control, you must maintain control allocation. If control is diminished (all two arm RAR, some multiple arm RAR) you can have problems. Modern platform trials maintain control allocation, and thereby achieve improved performance.

(27/n) References page 1
Korn Freidlin 2011
ncbi.nlm.nih.gov/pubmed/21172882
Connor et al 2013
ncbi.nlm.nih.gov/pubmed/23849147
Lewis et al 2013
ncbi.nlm.nih.gov/pubmed/23514753
Wason Trippa 2014
ncbi.nlm.nih.gov/pubmed/24421053
Thall et al 2015
ncbi.nlm.nih.gov/pubmed/25979922

(28/END) References page 2
Berry et al 2016
ncbi.nlm.nih.gov/pubmed/26768569
Wathen Thall 2017
ncbi.nlm.nih.gov/pubmed/28982263
Ventz et al 2018
ncbi.nlm.nih.gov/pubmed/30646308
Alexander et al 2018
ncbi.nlm.nih.gov/pubmed/28814435
Viele et al 2020
ncbi.nlm.nih.gov/pubmed/31630567