To *animal researchers*, a must-read on a sensitive topic rdcu.bs/cfsk6: statistical power of #animalresearch and of the number of animals used. A thread of why-do-I-like-this-paper or watch a 10-min video bit.ly/2M0EyNm @NatureNeuro go.nature.com/3dopwwl
Disclaimer: own production with amazing collaborators @sarabdjitsingh@MathiasVSchimdt@KorosiAniko@z_baram@JelleKnop@JessicaLBolton@joeribordes & twitterless others
Red is bad, green is good. At best 12.5% of animal experiments are estimated to be sufficiently prospectively powered. ‘At best’ because the estimate is before any subsequent experimental bias, p-hacking/fishing, multiple testing… 
Sample size from >450 publications (sys review), effect sizes estimated from literature (25,50,75th percentile) – basically the fig shows what’s reasonable to expect with a new study.
The question then becomes: how to increase power without increasing sample size to unrealistically high levels? The solution we propose is to use data of previous *control* animals for the new experiment.
This is an application of Bayesian priors. For an intro to Bayes, I recommend this paper from @RensvdSchoot nature.com/articles/s4358…
With a simulation study, we show that data of previous experiments can reduce the number of animals required, or alternatively increase statistical power (or a combination). Of note, the red diamond is the median N we use now (for 2 groups).
“Great, but that’s fake data. I will believe it only if I see it on real data”. Unfortunately, “real data” large enough to validate this does not exist, since that’s literally the problem.
We founded a consortium, RELACS (and it just took ~2y!). 10 laboratories shared their data on a specific experiment, i.e. the effects of limited nesting on OIL task. The dataset was large enough to validate the methodology.
It works! From the top: (a) analysis without prior provides the same result as a Welch t-test; (b) if ncon is decreased, the study becomes underpowered; (c) but this can be rescued if a prior from (unrelated) published literature is introduced.
If this method were implemented now, without increasing the total number of animals, prospective statistical power for large effect sizes would increase from 12.5% to 68.75.
You probably have more questions, like “what about batch effects?” or “What if my previous animals are different from my new ones?” Read the discussion of the paper. I purposely do not intend to summarize this issue, as it deserves appropriate attention rdcu.bs/cfsk6
This method works better for outcomes that are more commonly used. It is by no means a unique answer to the complex issue of animal experimentation, but some that can just be implemented …. Now. Then, why not?
To facilitate the implementation of the method, we developed a user interface. I LOVE #rstats and #shiny utrecht-university.shinyapps.io/repair/
I like this paper so much because (1) it’s full of #openscience. Project, data, code available here osf.io/wvs7m/
(2) It’s full of #WomeninSTEM! 3 out of the 4 main authors, and 12 out of 21 consortium authors. (3) I criticize some of the work of my own professor – and she supported me anyway. Isn’t she the most outstanding example?
(4) Sometimes science can “just work”. I had an idea, my supervisors supported me to bring it forward, we wrote a paper, got critical reviewers’ feedbacks, improved the quality of the work, rewritten it with help from the editor, accepted.
(5) As agreed by Marian “our names look beautiful next to the Nature Neuroscience logo” @NatureNeuro
I dream of the day when animal research won’t be necessary anymore, but this might well be not in my lifetime. For now, we can be creative with analytical techniques to reduce animal experimentation.
It’s ok if your feelings race between “oh god” and “oh god”, with a twist of “I am actually not that surprised”. I had the same when I first looked at the figure.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
