Do you think the news is bad at presenting stats?
@TomChivers and I do, so we made a "statistical style guide" for journalists.
If you think journalists should follow the guide then please join our campaign (just name and e-mail) here...
howtoreadnumbers.com
@TomChivers and I do, so we made a "statistical style guide" for journalists.
If you think journalists should follow the guide then please join our campaign (just name and e-mail) here...
howtoreadnumbers.com
Our Statistical Style Guide:
1) Put numbers into context.
Ask yourself: is that a big number? Don't just say "361 cyclists were killed on London roads between 1993 and 2017". We also need to know how many cycle journeys were made in London over this period.
1) Put numbers into context.
Ask yourself: is that a big number? Don't just say "361 cyclists were killed on London roads between 1993 and 2017". We also need to know how many cycle journeys were made in London over this period.
2) Give absolute risk, not just relative.
If you tell me that eating burnt toast will raise my risk of a hernia by 50 per cent, that sounds worrying. But unless you tell me how common hernias are, it’s meaningless. Let readers know the absolute risk.
If you tell me that eating burnt toast will raise my risk of a hernia by 50 per cent, that sounds worrying. But unless you tell me how common hernias are, it’s meaningless. Let readers know the absolute risk.
3) Check whether the study you’re reporting on is a fair representation of the literature.
If you’re reporting on a new study that finds that red wine is good for you, it should be presented in the context that any individual study can only be part of the overall picture.
If you’re reporting on a new study that finds that red wine is good for you, it should be presented in the context that any individual study can only be part of the overall picture.
4) Give the sample size of the study – and be wary of small samples.
A drug trial which has 10,000 subjects should be robust against statistical noise or random errors. A psychological study looking at 15 undergraduates is likely to give spurious results.
A drug trial which has 10,000 subjects should be robust against statistical noise or random errors. A psychological study looking at 15 undergraduates is likely to give spurious results.
5) Be aware of problems that science is struggling with, like p-hacking and publication bias.
Journalists can’t be expected to be an expert in every field, but being aware of these issues can help when you come across a new "surprising" study.
Journalists can’t be expected to be an expert in every field, but being aware of these issues can help when you come across a new "surprising" study.
6) Don’t report forecasts as single numbers. Give the confidence interval and explain it.
A forecast of 2.4 per cent can sound more accurate and scientific than it is if you don’t mention that their 95 per cent uncertainty interval is between -1.1 per cent and +5.9 per cent.
A forecast of 2.4 per cent can sound more accurate and scientific than it is if you don’t mention that their 95 per cent uncertainty interval is between -1.1 per cent and +5.9 per cent.
7) Be careful about saying or implying that something causes something else.
Lots of studies find correlations between one thing and another – between drinking fizzy drinks and violence, for instance. But the fact that two things are correlated doesn’t mean its causal.
Lots of studies find correlations between one thing and another – between drinking fizzy drinks and violence, for instance. But the fact that two things are correlated doesn’t mean its causal.
8) Be wary of cherry-picking and random variation.
If you notice that something has gone up by 50 per cent between 2010 and 2018, have a quick look – if you’d started your graph from 2008 or 2006 instead, would the increase still have looked as dramatic?
If you notice that something has gone up by 50 per cent between 2010 and 2018, have a quick look – if you’d started your graph from 2008 or 2006 instead, would the increase still have looked as dramatic?
9) Beware of rankings.
Has Britain dropped from the world’s fifth-largest economy to the seventh? Is a university ranked the 48th best in the world? What does that mean? Depending on the underlying numbers, it could be a big deal or it could be irrelevant.
Has Britain dropped from the world’s fifth-largest economy to the seventh? Is a university ranked the 48th best in the world? What does that mean? Depending on the underlying numbers, it could be a big deal or it could be irrelevant.
10) Always give your sources.
This is key. Link to, or include in your footnotes, the place you got your numbers from. If you don’t, you make it much harder for people to check the numbers for themselves.
This is key. Link to, or include in your footnotes, the place you got your numbers from. If you don’t, you make it much harder for people to check the numbers for themselves.
11) If you get it wrong, admit it.
Crucially – if you make a mistake and someone points it out, don’t worry. It happens all the time. Just say thank you, correct it, and move on.
Crucially – if you make a mistake and someone points it out, don’t worry. It happens all the time. Just say thank you, correct it, and move on.
We don't expect everyone to agree with everything on the list and that's fine.
The ultimate aim is to get news institutions to think about how they present numbers and develop their own style guide!
You can view the style guide in full here
howtoreadnumbers.com/stats-style-gu…
The ultimate aim is to get news institutions to think about how they present numbers and develop their own style guide!
You can view the style guide in full here
howtoreadnumbers.com/stats-style-gu…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
