Thread by @Kristin_Hook

👋 I’m a scientist who's spent the last decade of my life doing research. Everyone reading or watching the news lately has been exposed to more science & data than they could have ever imagined (or hoped for), AND THE TIME HAS COME TO GET THE WORD OUT ABOUT HOW SCIENCE IS DONE!

Most scientific research is funded by federal agencies (@NSF, @NIH, @USDA, etc). These agencies are allotted funding by Congress every year, and they then decide how to dole out that funding to researchers.

Researchers like me do not just get handed money to do our research. We have to apply for grants to do so.

These grants are REALLY competitive!!

For example, the NSF and NIH fund around 30% of proposals that they receive.

The competition for research funding is fierce and has gotten more competitive over the last two decades because there has been an increase in the number of scientists/investigators who want to do research but a fixed budget to pay for that research. 👇

As a scientist, part of our job is to secure funding to actually do our research. This is NOT an easy task!!

You have to figure out where the gaps in knowledge are in your field, why they are important, & think about clever studies & experiments to adequately address those gaps

Then you write a long and detailed project proposal explaining the problem, how you will solve that problem with your study, why it's important, what you will learn, how funding will help, and a budget outlining what you will spend that money on.

THIS IS A TON OF WORK!

Once you have successfully submitted your grant to an agency, you wait months to find out if it was funded or not. From the agency side, they create panels of expert reviewers who look over your proposal and decide whether it is worth funding or not.

If it does get funded, then luck you!!!! You then can go on to do the research!!!

Once you’ve carried out the research study (side bar: these sometimes take YEARS!), then you have to write it up for publication to communicate your results.

So you write a manuscript describing your study – your initial question and background for the study, what you did, what you found, and what it all means. Then you submit it to a peer-reviewed journal where you hope it will get published.

Journal editors can either say “Hell yes, we want this” or “Hell no, we will never publish this”.

If they are interested, they send it off to reviewers - other expert scientists, often in your field, who have volunteered to evaluate the validity of your study and its findings.

Depending on the journal, the identities of the reviewer and/or author may be concealed. These "blind" review practices are good so that you can reduce bias and allow reviewers to honestly review scientific submissions and suggest earnest ways to make them better.

It is these reviewers who determine the quality of the study and whether it warrants publication. They suggest ways to improve it - in small ways, like a spelling error or confusing wording in a sentence, or in big ways, like YOUR RESULTS DO NOT WARRANT THE CONCLUSIONS YOU MADE.

If it is accepted, it is at this stage that you revise your paper, incorporating feedback and suggestions before it is published and can be read and shared by others.

This whole process, though tedious and cumbersome (and annoying if you have to submit your paper multiple times to multiple journals), is NECESSARY!!

What it means is that published works that are peer reviewed are vetted works of science.

Is it a perfect system? No, but it is a good system for making sure crap science doesn’t get put out there.

Science is a process that builds on itself, so one bad paper can become a seed for other bad scientific studies, affecting the whole scientific enterprise. That is why it's SUCH A BIG DEAL when one of our own makes up data. We want nothing of these scientists who lack integrity.

So for those of you who are not scientists: when you see or hear about results from a scientific study that have not been peer-reviewed or published, YOUR ANTENNAE SHOULD GO UP!

Early studies that have not been peer-reviewed have not been vetted and mean that we cannot conclusively rely on their results. Just because someone did a study does not mean it was good. Only the review process can determine that.

UPDATE: as others have rightly pointed out, it's also true that just because a paper was reviewed does NOT mean it is a good study either.

The context for this tweet is that there are a lot of folks who are making claims based on data that are preliminary and that have not been reviewed. I wanted to clear up how the process is done so that you can better evaluate those claims.

EDIT: I should have clarified that funding varies - with 30% being the best-case scenario, but the odds being any where from 10-30%, dependent on the research and funding source.

Here are some recent data on rates:

NSF: dellweb.bfa.nsf.gov/awdfr3/default…

NIH: nexus.od.nih.gov/all/2019/03/13…