Thread by @callin_bull

1. I recently tweeted about a particularly poor piece of science reporting in the science/tech news site @BigThink. In that article, they describe a new study as showing that spending two hours a week in nature is essential for happiness.
bigthink.com/surprising-sci…

2. In our course, we encourage our students to question as strongly those claims that support their beliefs as those that challenge them.

I believe myself that time in nature improves well-being, so in the spirit of following my own advice, let’s look closely at this story.

3. The @BigThink story oversells the study. First of all, this is an observational study that establishes association, not causation. It could be that time in nature causes a sense of well-being. Or it could be that a sense of well-being causes people to spend time in nature.

4. E.g. when I (CB) was struggling with depression, getting out into nature felt overwhelming difficult.

Or it could be some common cause (wealth, physical health, etc.) causes both well-being and time in nature. The authors attempt to control for some of these, but it’s hard.

5. Worse still, @BigThink makes the prescription that you should spend time in nature to be happy. Now, that might entirely true. But the study in question does not show this at all. If a study only shows an association, you can’t use it to make prescriptive recommendations.

6. Second, @Bigthink makes a bizarre claim that 120 minutes / week is somehow a critical threshold. Both self-reported health and well-being increase monotonically through 240 minutes with time spent in nature. How could you claim the data below as evidence for a threshold?

7. Third, the @BigThink story claims that 120 minutes/week is *essential* for happiness. Simply read the title of the research paper—“Spending at least 120 minutes a week in nature is *associated with* good health and well-being”—and you see that BigThink has oversold it.

8. Or you could look at the data. Of the participants spending 0 minutes/week in nature, 68.5% report good health and 55.7% report high subjective well-being. That shows clearly that time in nature is *not* essential, even if it may be helpful!

9. It’s interesting to do a post-mortem and figure out where the bullshit arose. We have:

+ the original research paper nature.com/articles/s4159…

+ the press release from Exeter University exeter.ac.uk/research/newsa…

+ the @BigThink story.
bigthink.com/surprising-sci…

10. (We also have a popular account of the study written in @ConversationEDU by the first author. In my opinion this piece is appropriately cautious and accurate, by and large, and I won't discuss it further here.) theconversation.com/spending-two-h…

11. The research paper is clear that this is an observational study that does not establish causality and cannot be used to derive prescriptive recommendations.

12. But everything goes to pieces in the press release. It’s not solely the fault of the press office; some of the blame lays squarely with the paper's authors.

13. Author Terry Hartig endorses the prescriptive value of the study: "The current findings offer valuable support to health practitioners in making recommendations about spending time in nature to promote basic health and wellbeing, similar to guidelines for weekly physical.”

14. Unsurprisingly, this becomes a major take-home message of the @BigThink article, which concludes with the following text:

15. Now on to the issue of the 120 minute threshold. To my surprise, this seems to have arisen in the research paper itself.

First of all, the study participants are grouped into categories:

0 min/week,
1-59 min/week,
60-119 min/week,
120-179 min/week, etc.

16. So the resolution of the study is severely limited. If there is a critical threshold, the study couldn’t tell us whether it is at 100 minutes or 140 minutes rather than 120, for example.

17. Moreover, the researchers report a significant difference between people with 0 minutes and people with 60 or more minutes, using their raw data. It is only as they move to "adjusted models" that the difference appears at the 120 minute mark instead 60. And they write:

18. So there's a steady positive relationship up to 120 minutes, and declining benefits afterwards. That's hardly a threshold. Yet in the discussion we read that:

19. Are the authors right or wrong here? I personally think that they may have conflated absence of statistical significance below 120 min with absence of an effect below 120 min. But it's a minor sin if one at all; they state clearly that the threshold is merely a hypothesis.

20. The point is simply that this particular aspect of the @BigThink article traces back all the way to the original scientific manuscript. You can hardly blame @BigThink for that.

21. Finally this business about time in nature being essential for well-being. That's not in the research article or the press release. It first appears in the @BigThink story. Such is actually somewhat unusual actually.

22. When health news is oversold, which it often is (see e.g. ncbi.nlm.nih.gov/pmc/articles/P…), the error typically arises in a press release rather than subsequently: bmj.com/content/349/bm…

23. This is an extremely important point. If we care about public trust in science, we need to earn it. And that requires working with our press offices to make sure that our findings are not being oversold.

24. We saw that overselling at the press release level in this particular case, as we went from a paper that cautioned against prescriptive interpretation to a press release (and author quotes!) that encouraged such.

25. My aim with this thread is not to pick on this particular study or even on this particular article. It is to illustrate the way in which science news—and health news in particular—is distorted as it travels from research paper to popular scientific story.

26. In this case, we see that there's not one single locus of blame. This problems arise at every stage: research paper, press release, and popular article.

If we want to communicate science to the public, we need to work together across these levels to do better.