My editorial paper entitled, "No raw data, no science: another possible source of the reproducibility crisis". When I feel the results are too beautiful, I requested raw data before sending it out for review. 40 out of 41 did not send me the right data.
molecularbrain.biomedcentral.com/articles/10.11…
molecularbrain.biomedcentral.com/articles/10.11…
21 of them were just withdrawn without providing raw data. 19 of them were rejected due to insufficient raw data (ex. just one sample per group provided) and/or to mismatches between raw data and the results shown. Mismatches are mostly huge, not subtle.
14 out of those 40 that were withdrawn or rejected were published in other journals (impact factor from 2.2 to 4.7; mean: 3.37). 12 journals had a policy requiring or encouraging raw data sharing upon request and so I requested raw data. None out of 12 gave me what I requested.
I cannot help suspecting that some (or many) of them, from the beginning, did not have the right raw data that match the results shown.
We are in a serious situation. Science is supposed to be based on data/evidence. But, what if there is no data or no evidence fo the existence of raw data supporting the results of scientific papers? It is like a tower built on the sand.
Researchers are under pressure to publish papers in ruthless competition. What if researchers who produce beautiful results based on non-existing data win against the ones who struggle with true data? Unfortunately, that is what is happening in science nowadays, I am afraid.
So, my proposal is that journals and funding agencies mandate the authors to publish raw data on or before publication of the paper. Funding agency and institutions should cooperate to establish infrastructure to realize this.
A few people warned me that the editorial may make our journal look a low grade journal. I don't think that this is specific to our journal, or journals with standard impact factor.
I recently requested raw microscope images in three papers in two high impact factor journals (one's IF is more than 20 ( @cellstemcell ), and one is more than 30 (
@naturemedicine
)). None out of 3 gave me any raw data.
@naturemedicine
)). None out of 3 gave me any raw data.
@CellStemCell @NatureMedicine The authors of two papers declined to do so saying that just did live counting of immuno-positive cells and they did not save image files (They say that they threw away raw image files).
@CellStemCell @NatureMedicine One declined to do so, saying that they are conducting the analyses I proposed to do with the images and so cannot provide the images.
@CellStemCell @NatureMedicine N is just 3 and so I am not absolutely sure. But I guess that situation is more or less similar in high IF journals.
@CellStemCell @NatureMedicine I'd like to draw attention to my editorial. People who are involved in #opendata, #openscience, and integrity in science, @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata #NIHData #openresearch2020, please take a look & retweet.
@CellStemCell @NatureMedicine @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata Here is an article by THE on my editorial.
I believe that "Journal transparency index" by @OSFramework, considering data transparency as an index to evaluate journals, should be ‘alternative’ to current IF, as an article linked here proposes.
timeshighereducation.com/news/editors-i…
I believe that "Journal transparency index" by @OSFramework, considering data transparency as an index to evaluate journals, should be ‘alternative’ to current IF, as an article linked here proposes.
timeshighereducation.com/news/editors-i…
@CellStemCell @NatureMedicine @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata Absolutely right. Every lab should do this.
@CellStemCell @NatureMedicine @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata I hope editors of other journals take this seriously, too.
@CellStemCell @NatureMedicine @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata Getting a lot of feedbacks. Thanks. I want journal editors and funding agencies take a look at these comments, too.
@CellStemCell @NatureMedicine @OSFramework @NIHDataScience @ODIHQ @odsc @opendatacharter @OpenDataWatch @EU_opendata Regarding the importance of sharing raw data, check out our PNAS paper, as an example, that reached totally 'opposite' conclusion of the original study by re-analysis of exactly same raw data.
