Facebook research: despite improvements in statistical techniques for causal inference from observational data, and even with massive statistical power, observational methods fail to capture causal effects revealed by direct experiment kellogg.northwestern.edu/faculty/gordon…
"Using data from 15 US advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions..."
" The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables"
"commonly used observational approaches based on the data usually available in the industry often fail to accurately measure the true effect of advertising."
Related: Deming, data and observational studies (2011)
rss.onlinelibrary.wiley.com/doi/epdf/10.11… out of 52 claims about nutrition based on observational studies, none replicated in randomised trials (and 5 trials showed effects in the opposite direction)
rss.onlinelibrary.wiley.com/doi/epdf/10.11… out of 52 claims about nutrition based on observational studies, none replicated in randomised trials (and 5 trials showed effects in the opposite direction)
