We’re now passing over to @DrGWheeler to walk us through software for adaptive designs!
Sorry, attempt 2 (@JMSWason not too experienced with adding gifs!)
Sorry, attempt 2 (@JMSWason not too experienced with adding gifs!)
In 2020 Michael Grayling @UniofNewcastle & @DrGWheeler @ucl published a review of software for #AdaptiveDesigns in clinical trials.
Lack of available software is a known barrier to using better trial designs in practice. Q: “How bad is this problem?” 1/9
journals.sagepub.com/doi/full/10.11…
Lack of available software is a known barrier to using better trial designs in practice. Q: “How bad is this problem?” 1/9
journals.sagepub.com/doi/full/10.11…
Our research questions were:
1. How many articles proposing new adaptive designs included code with the paper, or a link to code available elsewhere?
2. Which adaptive design approaches and features are well supported by current software, and (importantly), which are not? 2/9
1. How many articles proposing new adaptive designs included code with the paper, or a link to code available elsewhere?
2. Which adaptive design approaches and features are well supported by current software, and (importantly), which are not? 2/9
For Q1, we reviewed 4,123 articles published from 2013-2017 across 31 journals. Of these, 247 articles from 26 journals were considered eligible.
174 (70%) articles provided no code relating to the proposed method.
Figure 1 shows code provision status by journal. 3/9
174 (70%) articles provided no code relating to the proposed method.
Figure 1 shows code provision status by journal. 3/9
We found that even if journals had a compulsory code provision policy, this wasn't a guarantee that code was available.
Figure 2 shows code provision by journal policy. Of 66 publications in Statistics in Medicine, which required authors to supply code, 51 (77%) did not. 4/9
Figure 2 shows code provision by journal policy. Of 66 publications in Statistics in Medicine, which required authors to supply code, 51 (77%) did not. 4/9
129 (52%) articles stated what software was used, but 60 (47%) such articles did not make their code available. #R was by far the most popular software used (107 (91%) articles, 16 of which used R with another program, or provided code in R and another language). #Rstats 5/9
For Q2, we searched software libraries to see which adaptive designs were available. Of 122 eligible records, most were for group sequential methods in phase II & III trials. Few were available for sample size adjustment, adaptive randomization, or biomarker-based methods. 6/9
Finally, we looked at the quality of support around each record. Of the 122 records, 95 (78%) provided help files, but only 42 (34%) provided a detailed guide or vignette. Only 16 records (13%) contained “well annotated” code. 7/9
Summary:
1. Most articles proposing adaptive designs for clinical trials don’t provide code or software;
2. Adaptive trial resources focus on group sequential methods for later phase trials;
3. Help files are often provided with code, but annotation of code is poor. 8/9
1. Most articles proposing adaptive designs for clinical trials don’t provide code or software;
2. Adaptive trial resources focus on group sequential methods for later phase trials;
3. Help files are often provided with code, but annotation of code is poor. 8/9
Finally, two tips for adaptive design researchers:
1. Learn some basic #R, as this is what most packages are written in. Plus, you can take an existing package and tweak that code for your own work.
2. Please... PUBLISH👏YOUR👏CODE!
Thanks for reading! #AdaptiveDesigns 9/9
1. Learn some basic #R, as this is what most packages are written in. Plus, you can take an existing package and tweak that code for your own work.
2. Please... PUBLISH👏YOUR👏CODE!
Thanks for reading! #AdaptiveDesigns 9/9
• • •
Missing some Tweet in this thread? You can try to
force a refresh
