If you are about to tweet something in vague support of trials, STOP THERE!
Before you tweet that vague support, read this handy checklist of what might/will happen and ask yourself, ‘Is this how I want to spend today?’
Before you tweet that vague support, read this handy checklist of what might/will happen and ask yourself, ‘Is this how I want to spend today?’
1. A person-bot who always triumphally exclaims ‘real-world evidence’ will triumphally exclaim ‘real-world evidence’ (+hashtag #RWE), as though trials use fake-world people.
2. Someone will remind you that ‘You know we managed to do smoking causes lung cancer without randomisation or blinding, right? Still think trials are useful? Not so smart now are ya!’
3. Someone will say something about generalisability and it’ll take ages to work is if it’s the reasonable or unreasonable one or something totally new.
4. Someone will note the possibility that an unobserved covariate may be imbalanced by chance, conveniently forgetting that this is exactly why we have standard errors, and things will descend into the Cartwright and Deaton episode and the Kraus debacle.
5. Someone will forget about how confidence intervals and p-values relate and people will announce which side they are on, like ‘I’m more of a confidence interval person’, thinking that makes them Bayesian.
6. Someone will say to forget about quantifying uncertainty; the estimate is the effect (@jd_wilko)
6. Some conditional vs. marginal estimands stuff will go off and someone will confuse it with adjusted vs. unadjusted estimators.
7. The ‘in-the-pub characters’ who always deliver their very own one-liner turn up and deliver their one-liner [redacted in case these folks follow me… except ‘healthy human–time matrix’ and ‘have you heard of sequential analysis’ which are too good to omit]
8. [I’m out… add your favourite nuggets below!]
If you think you can get through that checklist, cool; you might learn new things (sometimes it’s just ‘a new way to misunderstand or misrepresent trials’)!
After all the bantz, remember to reflect on whether anyone has actually come up with a practical proposal that resolves all the criticisms and rivals trials for doing causal inference.
Trials aren’t perfect (we never say they are!), but until we have a clear improvement, here’s a statement of ‘vague support’ that I’m on board with*
*GLMs with canonical link fn are also magical but that’s another thread for another day
*GLMs with canonical link fn are also magical but that’s another thread for another day
