(1/12) I receive emails warning about hackers targeting COVID data. Huge issues..it’s a crime and a danger to trial integrity. But also begs another discussion…if your loved one could be randomized but you could just select the current best arm, how much is gained? 
(2/12) Your timing matters. The information you could use to select an arm accumulates over time.
The design matters. Some designs treat patients within the trial better.
The design matters. Some designs treat patients within the trial better.
(3/12) What is “the cost of randomization”? In a mortality trial, we might define it as the difference in survival probability between being randomized versus just being allowed to choose the current observed best arm.
(4/12) Example, 5 arm N=200 trial. In truth, unknown to you, the survival rates are 30, 40, 50, 60, and 70 percent. Equally randomized patients always have a 50% percent of survival (average across the arms). It doesn’t matter which patient you are, first or last.
(5/12) Suppose you could be the 200th patient or just select an arm. Randomized, you have a 50% chance of survival. But suppose you could just pick the best observed arm? Your chance of survival elevates to 67%. The cost of randomization, 17%, is quite high.
(6/12) This graph shows the probability of survival for randomized patients (always 50% percent) and “VIPs” who can just pick the best arm. The VIPs chances depend on when they arrive. The more information VIPs can use, the better.
(7/12) We can decrease the cost of randomization through adaptive design. Instead of fixed equal allocation, let’s employ response adaptive randomization (arm dropping is also possible). Patients late in the trial are more likely to be randomized to better arms.
(8/12) This graph shows the probability of survival for each randomized RAR patient. Coincidentally but importantly, their survival probability tracks the VIPs from a fixed trial. But they are randomized and contribute to future knowledge.
(9/12) Did we harm the knowledge gained by the study by using RAR? No. We actually improved it. The equal randomization trial has an 80% chance of identifying the correct arm (with 70% survival). The RAR trial has an 87% chance of identifying the best arm.
(10/12) Have we eliminated the cost of randomization? Not quite. While survival probabilities of randomized RAR patients track equal randomization VIPs, you could greedily consume the greater information from the RAR trial.
(11/12) The graph below adds the survival probabilities for “RAR VIPs”. They generally run 1-3% higher than randomized patients. The cost of randomization is therefore not eliminated, but dramatically reduced compared to equal randomization.
(12/12) These kind of calculations quantify many ethical arguments in equipoise and are relevant to experiments conducted directly in a healthcare setting (comparative effectiveness, etc.). We want to obtain knowledge while asking the minimum of patients to generate it.
