What’s up with RCTs of #exercise for people w #cancer?
Introducing OREO!
*Outcome Reporting bias in Exercise Oncology trials*
@BenSinghPhD @CiaranFairman @Katebolam @jfc_science @Rosie_Twomey @dnunan79 @IMLahart
Preprint: medrxiv.org/content/10.110…
#openaccess
THREAD🧵1/n
Introducing OREO!
*Outcome Reporting bias in Exercise Oncology trials*
@BenSinghPhD @CiaranFairman @Katebolam @jfc_science @Rosie_Twomey @dnunan79 @IMLahart
Preprint: medrxiv.org/content/10.110…
#openaccess
THREAD🧵1/n
There are now hundreds of RCTs in #exerciseoncology & many systematic reviews.
These underpin guidelines & calls for exercise to be part of cancer care.
#exerciseismedicine 2/n
These underpin guidelines & calls for exercise to be part of cancer care.
#exerciseismedicine 2/n
However, in many disciplines including #psychology & #oncology, there is evidence of selective reporting bias:
Reporting the results of some outcomes or analyses & not others, depending on the nature (e.g. significant p-value) or direction (e.g. positive) of the results.
3/n
Reporting the results of some outcomes or analyses & not others, depending on the nature (e.g. significant p-value) or direction (e.g. positive) of the results.
3/n
To minimize reporting bias & improve transparency, clinical trial registration is *required* by:
ICMJE: icmje.org/recommendation…
@WHO: who.int/publications/i…
@medwma
ncbi.nlm.nih.gov/pmc/articles/P…
...& has been for over a decade.
4/n
ICMJE: icmje.org/recommendation…
@WHO: who.int/publications/i…
@medwma
ncbi.nlm.nih.gov/pmc/articles/P…
...& has been for over a decade.
4/n
Specifically, *prospective* trial registration is required (this means *before* the first participant is enrolled).
To assess selective reporting bias in #exonc, Ben & Ian took on the HUGE task of comparing trial registrations with published manuscripts.
5/n
To assess selective reporting bias in #exonc, Ben & Ian took on the HUGE task of comparing trial registrations with published manuscripts.
5/n
The first important (somewhat unintended) finding is that ~half of RCTs of exercise for people with cancer are not compliant with international standards.
These are registered retrospectively, often many months after the trial start date.
6/n
These are registered retrospectively, often many months after the trial start date.
6/n
However, 31 preregistered trials & 78 published manuscripts were evaluated. A critical discrepancy between a trial reg & published manuscript is *switching* of the primary outcome (because it reflects the main aim of the RCT & is often used in the sample size calc).
7/n
7/n
Important to emphasize that discrepancies are only discrepancies if *undeclared*.
Deviations from preregistered protocols are a normal part of the scientific process. They simply need to be transparently reported
Don’t @ me about word count😊
(❤️supplementary files❤️)
8/n
Deviations from preregistered protocols are a normal part of the scientific process. They simply need to be transparently reported
Don’t @ me about word count😊
(❤️supplementary files❤️)
8/n
We found that about 25% of preregistered primary outcomes were *switched* (something else was reported as primary) in the published manuscript & some of those preregistered primary outcomes were omitted from manuscripts entirely.
9/n
9/n
The completeness of the trial registration varied & in more than one-third of regs, only the broad/general domain was preregistered (e.g. "quality of life") rather than the outcome score (e.g. FACT-General total score).
Vague descriptions = ⬆️analytical flexibility
10/n
Vague descriptions = ⬆️analytical flexibility
10/n
None of 78 published manuscripts from 31 eligible trials were free from incomplete/selective reporting
42% of outcomes in manuscripts were silently introduced (not included in trial regs) & 41% of outcomes in trial regs were silently omitted (not included in manuscripts)!
11/n
42% of outcomes in manuscripts were silently introduced (not included in trial regs) & 41% of outcomes in trial regs were silently omitted (not included in manuscripts)!
11/n
Selective reporting bias may inflate the Type I error rate to an unknown extent (increases the likelihood of false positives) & undermines not only the individual RCTs but also the systematic reviews they are included in.
Why is this a problem & what are the solutions?
12/n
Why is this a problem & what are the solutions?
12/n
One major incentive for outcome switching is publication bias:
The longstanding preference for positive findings & studies that find support for their hypothesis.
& the traditional academic reward/incentive system more generally, but that’s for another day😊
13/n
The longstanding preference for positive findings & studies that find support for their hypothesis.
& the traditional academic reward/incentive system more generally, but that’s for another day😊
13/n
For exercise to be recognized by key stakeholders, RCTs must be conducted (& reported) as intended: to minimize bias & provide high-quality evidence.
Prospective trial reg (w the required detail) is not optional, it is a scientific & ethical responsibility to participants
14/n
Prospective trial reg (w the required detail) is not optional, it is a scientific & ethical responsibility to participants
14/n
We call for funders, editors, article reviewers, researchers & others involved in exercise oncology to consider it mandatory.
YOU can evaluate the robustness of a finding when you read/review an #exonc RCT by checking the history of revisions in the clinical trial reg!
15/n
YOU can evaluate the robustness of a finding when you read/review an #exonc RCT by checking the history of revisions in the clinical trial reg!
15/n
It was fantastic to be a part of this expert team of scientists
(& as someone who has been a
@bengoldacre
fan since #badscience, it’s exciting to follow the lead of
@COMPare_Trials)
Finally, data, code & DETAILED material here: osf.io/4c8mb/
#openscience
🧵End
(& as someone who has been a
@bengoldacre
fan since #badscience, it’s exciting to follow the lead of
@COMPare_Trials)
Finally, data, code & DETAILED material here: osf.io/4c8mb/
#openscience
🧵End
Oops I forgot, we welcome feedback on the @medrxivpreprint preprint!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
