I love seeing journalists do a textbook job of calling bullshit on the misleading use of quantitative data.
Here's a great example. @RonDeSantisFL claimed that despite having schools open, Florida is 34th / 50 states in pediatric covid cases per capita. nbcmiami.com/news/local/des…
I don't know for certain what set off their bullshit detector, but one rule we stress in our class is that if something seems too good or too bad to be true, it probably is.
DeSantis's claim is a candidate.
Below, a quote from our book.
The very next paragraph of the book suggests what to do when this happens: trace back to the source. This is a key lesson in our course as well, and at the heart of the "think more, share less" mantra that we stress. Don't share the implausible online until you've checked it out.
So that's what investigative reporter Tony Pipitone @TonyNBC6 from South Florida's @nbc6 did.
I might have stopped there, and assumed the low rank was due to with testing rates and the fact that children are more often asymptomatic or mildly symptomatic. Tony didn't. He did something else that we also stress in our class: Beware of unfair comparisons. Again from the book:
It turns out that Florida does well relative to other states because it reports cases among children as those aged 0-14, instead of 0-17, 0-18, 0-19, or 0-20 as other states do.
In science, people tend to be most interested in positive results — a manipulation changes what you are measuring, two groups differ in meaningful ways, a drug treatment works, that sort of thing.
Journals preferentially publish positive results that are statistically significant — they would be unlikely to have arisen by chance if there wasn't something going on.
Negative results, meanwhile, are uncommon.
Knowing that journals are unlikely to publish negative results, scientists don't bother to write them up and submit them. Instead they up buried file drawers—or these days, file systems.
One of our key pieces of advice is to be careful of confirmation bias.
There's a thread going around about how the crop below is what happens when Twitter's use of eye-tracking technology to crop images is fed with data from a misogynistic society. I almost retweeted it. But…
…that story fits my pre-existing commitments about how machine learning picks up on the worst of societal biases. So I thought it was worth checking out.
While it would be fish in a barrel to drag this paper as a contribution to the pseudoscience of homeopathy, we'll largely pass on that here. More interestingly, this single paper illustrates quite a few of the points that we make in our forthcoming book.
The first of them pertains to the role of peer review as guarantor of scientific accuracy.
In our book we suggest that one never assume malice when incompetence is a sufficient explanation, and one never assume incompetence when an understandable mistake could be the cause.
Can we apply that here?
I bet we can.
A lot of cartographic software will choose bins automatically based on ranges. For example, these might be the 0-20%, 20-40%, 40-60%, 60-80%, and 80-100% bins.
As the upper bound changes over time, the scale slides much as we see here.
We've written several times about what we describe as Phrenology 2.0 — the attempt to rehabilitate long-discredited pseudoscientific ideas linking physiognomy to moral character — using the trappings of machine learning and artificial intelligence.
For example,, we've put together case studies on a paper about criminal detection from facial photographs...