1) Utilizing any external data in a clinical trial, including real world evidence, presents risks and benefits. These should be quantified and balanced by choosing an appropriate design (a tweetorial explaining this graph…) 
2) Simple example…dichotomous outcome, you have a novel therapy and resources for 60 patients. You also have a database of untreated patients showing a 40% response rate. How can you incorporate the database into the trial?
3) Statistically, the risks/benefits of external information depend on the difference between the 40% (database rate) and the true control rate for the clinical trial. This is often called “drift” in the literature, but has other names.
4) Drift is the result of many potential biases. The database may not have randomized patients, may have occurred at different sites, different population, different time, etc. These biases may be large or small.
5) When drift is small, the control rate in our clinical trial is near 40%. We are borrowing good information. The result is increased power, decreased type 1 error, and better estimates of treatment effect. All good.
6) When drift is large, you are borrowing biased (relative to your trial) information. You will get biased estimates and either increased type I error or decreased power depending on the direction of drift.
7) Example, suppose the real control rate for the clinical trial is 50%. You are borrowing 40% data. This pulls down the clinical trial control and makes it easier to win than it should be. You get inflated type 1 error.
8) How much is the power loss/type 1 error inflation? This depends on how aggressive you are. The more aggressive, the bigger the benefit for low drift but the bigger the costs for high drift. Let’s consider 4 options.
9) Design options (most to least aggressive borrowing)
A) single arm trial, assume control rate known 40% (OPC)
B) single arm trial, use 60 synthetic controls (24/60 responders)
C) randomize 2:1, augment control with 8/20 responders
D) randomize 1:1, ignore database
A) single arm trial, assume control rate known 40% (OPC)
B) single arm trial, use 60 synthetic controls (24/60 responders)
C) randomize 2:1, augment control with 8/20 responders
D) randomize 1:1, ignore database
10) The original graph (repeated here) shows the type I error and power for each of these designs, for control rates from 20% to 60% (40% is no drift). Power was computed assuming a 30% effect (e.g. 40% vs 70%, or 20% vs 50%, etc.)
11) The single arm designs suffer from very large type I error inflation for large drift, potentially exceeding 50% (combined with large power gains). If you are going to use these, the true control rate can’t exceed 40% by much.
12) The randomized designs have more limited type I error inflation (type 1 error is actually reduced for low to negative drift). But the randomized design have more limited power gains compared to no borrowing.
13) There is no “correct design”, but you should think about the range of likely drift. If you are SURE drift is minimal the single arm trials make sense. If you are concerned about more modest drift, an augmented design may be better balanced.
14) Here control rates from 30-45% are handled well by designs B and C (more power, controlled type I error). B has more power, but suffers quickly if the control rate exceeds 45%.
15) There is lots of advanced work in this area. For larger sample sizes you can estimate the drift and dynamically choose how aggressively to borrow, retaining many of the benefits while minimizing the risks.
16) These dynamic borrowing models can be calibrated to provide more benefit in a narrow range, or less benefit in a broad range. There are adaptive trials which start 2:1 and then may tend toward single arm (3:1, 4:1) or back to 1:1 adaptively depending on estimates of drift.
17) Summary…quantify the risks/benefits of different designs over a range of drift. Decide what range of drift you can confidently defend. Then choose a design appropriate for that range.
