@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko I'll try to answer this, but first I need to parse apart a few of @tomgoom points above, because this is a well-founded concern from the "RCT's don't mimic real clinical practice" line of thinking that gets this part right and part wrong. Please bear with me for a few Tweets.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko RCT's are intended to resolve (or at least "inform") questions where there is genuine uncertainty. So to revisit Tom's first example of studying a running injury with three potential treatment groups (load mgmt, strength, gait retraining)...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko ...I would argue that he's putting the cart before the horse here:
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko If this is part of your clinical decision making pathway, and you would have a strong inclination after this assessment that the patient in front of you clearly would benefit more from one of the three strategies - that patient is not suitable to be randomized.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko In a well-designed RCT only patients who should actually make it to the point of randomization are patients for whom some degree of uncertainty exists about which treatment is the best option.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko If a patient *clearly needs* gait retraining, they should not be randomized in this trial! Tom is correct that trials which make this mistake are indeed doomed to provide relatively little clarity on the questions they are designed to answer.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko So in this way, RCT's should be designed with the clinical practice/decision *very much* understood and in mind. If the initial patient assessment suggests a clear and obvious advantage of one of the treatment options for that patient - they shouldn't be in the trial!
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Trials should be including only patients for whom there is / would be genuine uncertainty about which option is best. Those are the people you actually need the trial to inform practice for!
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Now, there are a couple of special cases / tangents that I'd like to discuss in more detail.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko 1. A common concern is something like "but then trials are not widely generalizable." The need for "generalizability" is sometimes overblown, because what we really need from the RCT is answers to questions that exist. Again, using the example above, if...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko ...many patients with running injuries can be identified as good candidates for A, B, or C but there is a small minority for whom there is uncertainty, that small minority is the only group for whom the RCT *needs* to provide an answer.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko So chasing generalizability *sometimes* feels like a red herring. That may mean you only randomize patients who go through an initial screening for whom there is no clear-cut answer to which is best. Because in this way, the RCT is now mimicking clinical practice up to...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko ...the point of randomization. And if you imagine how those results would eventually be implemented in practice, it can & should mimic the RCT. Take patient through initial assessment, if clear advantage to any option before the point of uncertainty, use that.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko If no clear advantage after initial assessment, the RCT that we've just been talking about will (hopefully) give you an answer for the population of "patients that went through the initial assessment and didn't have a clear and obvious choice for what to do first"
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Special point 2: another common concern is related to heterogeneity of treatment effects. "Maybe some patients will benefit more from A while others will benefit more from B." Trials can be designed to study this!
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Things like adaptive-enrichment designs with pre-specified subgroups can be used to test consistency of treatment effect(s) across specific groups where you think there could be variability.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Sticking with our running injury example above, if you believe that there are one or more specific factors which *may* differentiate patients who benefit more from gait retraining versus load management, we can design a trial to randomized patients and then separately...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko ...test the treatment effect according to those factors. If there is no evidence that the treatment effect is variable, we would default to the overall pooled treatment effect estimate. But if there is a clear advantage of one choice in a subgroup, that can be teased out.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Special point 3: some people may also point out that these 3 things are not mutually exclusive (e.g. for some patients you may decide to do "load management" AND "gait retraining") - similarly, we can use factorial designs...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko ...to test the options alone or in combination with one another, if that is appropriate and mimics how you would do this in clinical practice.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko And now, @goldblatchikens, thank you for your patience, because I'm finally back around to your initial question:
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Trials do not always have to compare "treatments" against one another. They can also be designed to compare "pathways" or "strategies" against one another.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko I could use @tomgoom help in spelling this out for the specific case above, but I can imagine *something like* the following:
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Pathway 1: perform initial assessment.
If training load seems like the problem, use initial strategy of load management. Re-eval in X weeks. If symptoms not improved, use gait retraining.
If gait seems like problem, start with gait retraining. Re-eval in X weeks...
If training load seems like the problem, use initial strategy of load management. Re-eval in X weeks. If symptoms not improved, use gait retraining.
If gait seems like problem, start with gait retraining. Re-eval in X weeks...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko (hopefully you get the idea I'm driving at here - you can outline an entire "care pathway" and make that one of the randomized arms)
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Pathway 2: *all patients* start with load management. If no improvement after X weeks, perform gait retraining. Etc...
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Then at the end of the trial, your answer is not a simple comparison of "gait retraining" versus "load management" but a comparison of outcomes using the two "pathways" that were designed.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko In this way, trials *can* be designed to more appropriately mimic the clinical practice/pathway, while answering the question(s) of interest in a way that might feel more directly applicable to the daily clinical practice.
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko Using my little example that I started to spell out above, you would effectively be testing "does a targeted initial strategy based on an initial assessment work better than simply using an initial strategy of load management for all comers"
@goldblatchikens @tomgoom @MaartenvSmeden @jd_wilko And, as discussed, you can layer in other treatment(s) that would be considered using a factorial design, or make them part of the protocol in the treatment "pathways" you are comparing
