Thread by @_MiguelHernan

Randomization never ensures zero #confounding bias. It provides probabilistic bounds on confounding.

Therefore, by bad luck, the effect estimates from some perfectly conducted randomized #trials are substantially confounded. But we don't know which ones!

An eye-opening example:

In Denmark, 860 individuals were randomly allocated to either "intervention" or "control":
• No intervention was implemented
• Individuals were unaware of their allocation
• Mortality was higher in the intervention group with p=0.003

Keep this in mind when evaluating a trial.

Reference:
Vass M (PhD Thesis). Prevention of functional decline in older people. Faculty of Health Sciences, University of Copenhagen 2010, p.120.

Thanks to Mikkel Zöllner Ankarfeldt for bringing this example to my attention.

One explanation of this result is “bad luck”. Everybody knows that at least 5% of perfect #trials in the best journals are wrong—its 95% confidence interval doesn't include the true value.

But attributing the result to “bad luck”—and stopping there—misses an important point.

“Bad luck” means that some prognostic factors happened to be more common in the intervention group.

In observational studies, we call this imbalance confounding.

In randomized trials, the imbalance occurs by chance so we can call it "random confounding."
ncbi.nlm.nih.gov/pubmed/25687168

Thinking of bad luck as random confounding is helpful because confounding, whether random or not, can be reduced via adjustment for measured covariates.

That is, in randomized trials, we are better off ADJUSTING for prognostic factors that happen to be imbalanced between groups.

Yes, the Danish result is due to chance, but we are not completely helpless against chance: We could adjust for imbalanced covariates. Whether the study is observational or randomized is irrelevant.

More details in Chapter 10 of our Causal Inference book: hsph.harvard.edu/miguel-hernan/…