Thread by @kirstyjean

Today I received the worst review of a manuscript I’ve ever had. The TL;DR: great study, shame the results were negative. Here’s why that’s a crappy attitude - a thread (1/n)

For some context, we tested whether physiological stress influences a morphological trait in a lizard. Lots of reason to believe that it might, results in other studies have been mixed. So, totally worth investigating further!
(2/n)

We put together a really rigorous combination of field and lab studies including manipulations and found no evidence that stress affected this particular trait. We thought this was cool as it suggests that perhaps trade offs maintain this trait in the face of stress (3/n)

or you know what? Maybe stress just doesn’t influence some traits as much as you’d expect (the prevailing wisdom is that “stress = BAD”). So, we still thought our results were cool and valuable. Journal disagreed. (4/n)

I could understand a verdict of our negative results not being novel enough. That can be a fair assessment, and I’ve published enough negative results to not take that personally from middle- to high-tier journals (5/n)

But the verdict instead was that our results, being negative, don’t further our understanding. Y’all, that’s just BS and not how science works (6/n)

They went on to wring their hands about what a difficult decision this had been because they recognised the elegance of our study To loosely paraphrase because I don’t want to quote directly, “you did great and careful work, it’s too bad the results don’t reflect that” (7/n)

I don’t think I need to say that a study’s results do not reflect anything other than the truth. Well-designed studies don’t equal significant results; badly designed studies don’t equal negative results. THAT’S 👏 NOT 👏 HOW 👏 HYPOTHESIS 👏 TESTING 👏 WORKS! (8/n)

It gets even worse. The editor commiserated that we’d had the “misfortune” to pick the “wrong” study organism for this work. Because the results didn’t fit our hypothesis.

Once again for the people in the back: That’s not how science works!! (9/n)

We tested this Q in this species because a) we know how to manipulate levels of a stress hormone effectively in this species and b) because they have the trait of interest. We didn’t pick it because we thought it would give us positive results. And no one should do that (10/n)

Ok so now we all can agree on how wildly bad this editorial take is. Like, super bad. Not just because there are basic underlying ethical issues here with assuming people choose tests and organisms to get flashy significant results. (12/n)

I’m not going to do science to get positive results. I do science to further knowledge, which is what they say we don’t do. Here’s why that’s crap. ONLY PUBLISHING POSITIVE RESULTS DOES NOT ADVANCE SCIENCE - IT HOLDS IT BACK. (13/n)

This editorial mindset does not promote reproducibility. Instead it promotes confirmation bias - see this great blog from theoretical ecology on why this sucks theoreticalecology.wordpress.com/2014/01/22/sei… (14/n)

If we only see (and publish) “X significantly predicts Y” we are missing important information on when X does NOT predict Y, and why not - this is valuable, important, science-furthering knowledge and information. (14/n

Plus, if studies with neg results are consistently getting binned, we end up with people doing the same study over and over because they think it hasn’t been done. 1, there’s not enough funding to waste it like this, and 2, THAT IS NOT ADVANCING THE FIELD (15/n)

To wrap this up - your well-designed studies that wound up having negative results are IMPORTANT. They MATTER. They ADVANCE SCIENCE. I will for sure be submitting this elsewhere and will think hard before submitting to this journal again (16/16)

Current heart rate, probably can skip the gym today 👌 #ragetweetingsaveslives