The #covid19 pandemic has led to a flood of misleading science as researchers rush to disseminate findings without going through normal review processes. newscientist.com/article/mg2463…
With unverified results circulating widely in news outlets and social media, it pays to be able to recognise the telltale signs of a study that might be poor. Here are some potential warning flags: newscientist.com/article/224283…
1. Study published on a blog, preprint server or social media. It might be valid but be cautious because it is unlikely to have been reviewed by independent experts.
2. Study has only one author. This may indicate that the paper is an early exploration of an idea or a tentative new hypothesis that shouldn’t necessarily be taken too seriously.
3. The researchers are from a surprising field of study. All fields of research require specialist training and knowledge, but that doesn’t stop some people applying themselves to subjects they know little about.
4. The analysis is very fast. When studies are published within days or weeks of an event, or report results from a clinical trial that hasn’t finished yet, these first impressions should be treated with caution.
5. The study is very small. Medical studies that look at small numbers of people are less likely to have results that will stand up in further trials. There are no strict rules, but anything under 50 participants is highly tentative.
6. The trial has no placebo group. It isn’t always possible to design experiments to include a placebo group, but without one, it can be difficult to know for sure if the observed effects are meaningful.
7. The study reports a correlation or association. Many factors can be linked without one causing the other. In observational studies, factors such as age, wealth or sex may also have an effect on the issue in question.
