, 26 tweets, 9 min read Read on Twitter
1. I recently tweeted about a particularly poor piece of science reporting in the science/tech news site @BigThink. In that article, they describe a new study as showing that spending two hours a week in nature is essential for happiness.
2. In our course, we encourage our students to question as strongly those claims that support their beliefs as those that challenge them.

I believe myself that time in nature improves well-being, so in the spirit of following my own advice, let’s look closely at this story.
3. The @BigThink story oversells the study. First of all, this is an observational study that establishes association, not causation. It could be that time in nature causes a sense of well-being. Or it could be that a sense of well-being causes people to spend time in nature.
4. E.g. when I (CB) was struggling with depression, getting out into nature felt overwhelming difficult.

Or it could be some common cause (wealth, physical health, etc.) causes both well-being and time in nature. The authors attempt to control for some of these, but it’s hard.
5. Worse still, @BigThink makes the prescription that you should spend time in nature to be happy. Now, that might entirely true. But the study in question does not show this at all. If a study only shows an association, you can’t use it to make prescriptive recommendations.
6. Second, @Bigthink makes a bizarre claim that 120 minutes / week is somehow a critical threshold. Both self-reported health and well-being increase monotonically through 240 minutes with time spent in nature. How could you claim the data below as evidence for a threshold?
7. Third, the @BigThink story claims that 120 minutes/week is *essential* for happiness. Simply read the title of the research paper—“Spending at least 120 minutes a week in nature is *associated with* good health and well-being”—and you see that BigThink has oversold it.
8. Or you could look at the data. Of the participants spending 0 minutes/week in nature, 68.5% report good health and 55.7% report high subjective well-being. That shows clearly that time in nature is *not* essential, even if it may be helpful!
9. It’s interesting to do a post-mortem and figure out where the bullshit arose. We have:

+ the original research paper nature.com/articles/s4159…

+ the press release from Exeter University exeter.ac.uk/research/newsa…

+ the @BigThink story.
10. (We also have a popular account of the study written in @ConversationEDU by the first author. In my opinion this piece is appropriately cautious and accurate, by and large, and I won't discuss it further here.) theconversation.com/spending-two-h…
11. The research paper is clear that this is an observational study that does not establish causality and cannot be used to derive prescriptive recommendations.
12. But everything goes to pieces in the press release. It’s not solely the fault of the press office; some of the blame lays squarely with the paper's authors.
13. Author Terry Hartig endorses the prescriptive value of the study: "The current findings offer valuable support to health practitioners in making recommendations about spending time in nature to promote basic health and wellbeing, similar to guidelines for weekly physical.”
14. Unsurprisingly, this becomes a major take-home message of the @BigThink article, which concludes with the following text:
15. Now on to the issue of the 120 minute threshold. To my surprise, this seems to have arisen in the research paper itself.

First of all, the study participants are grouped into categories:

0 min/week,
1-59 min/week,
60-119 min/week,
120-179 min/week, etc.
16. So the resolution of the study is severely limited. If there is a critical threshold, the study couldn’t tell us whether it is at 100 minutes or 140 minutes rather than 120, for example.
17. Moreover, the researchers report a significant difference between people with 0 minutes and people with 60 or more minutes, using their raw data. It is only as they move to "adjusted models" that the difference appears at the 120 minute mark instead 60. And they write:
18. So there's a steady positive relationship up to 120 minutes, and declining benefits afterwards. That's hardly a threshold. Yet in the discussion we read that:
19. Are the authors right or wrong here? I personally think that they may have conflated absence of statistical significance below 120 min with absence of an effect below 120 min. But it's a minor sin if one at all; they state clearly that the threshold is merely a hypothesis.
20. The point is simply that this particular aspect of the @BigThink article traces back all the way to the original scientific manuscript. You can hardly blame @BigThink for that.
21. Finally this business about time in nature being essential for well-being. That's not in the research article or the press release. It first appears in the @BigThink story. Such is actually somewhat unusual actually.
22. When health news is oversold, which it often is (see e.g. ncbi.nlm.nih.gov/pmc/articles/P…), the error typically arises in a press release rather than subsequently: bmj.com/content/349/bm…
23. This is an extremely important point. If we care about public trust in science, we need to earn it. And that requires working with our press offices to make sure that our findings are not being oversold.
24. We saw that overselling at the press release level in this particular case, as we went from a paper that cautioned against prescriptive interpretation to a press release (and author quotes!) that encouraged such.
25. My aim with this thread is not to pick on this particular study or even on this particular article. It is to illustrate the way in which science news—and health news in particular—is distorted as it travels from research paper to popular scientific story.
26. In this case, we see that there's not one single locus of blame. This problems arise at every stage: research paper, press release, and popular article.

If we want to communicate science to the public, we need to work together across these levels to do better.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Calling Bullshit
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!