,
4 tweets,
2 min read
Read on Twitter
Reading a paper emphasising diffs between two groups (n=40k each) on a cognitive task. They had near identical mean accuracy 75.2 vs 75.3, Cohen's d was <.2 and the p value was 0.000000006. Remember kids, bigger often just makes the trivial significant which is never better.
I should also add that the task was pretty weak to begin with. I worry that the push towards online testing and large sample sizes is going to lead to a lot of meaningless, but statistically significant, cog. neuro for a while. Just as underpowered samples did.
Once editors and reviewers get over the "you tested how many???” appeal of these papers, one way forward is to determine a priori the 'smallest effect size of interest' as recommended by @lakens.
And here's an example of a paper that compares an overpowered population sample to an optimally powered smaller sample and uses effect sizes throughout to draw the distinction between significant and meaningful: journals.plos.org/plosone/articl…
