,
7 tweets,
2 min read
Read on Twitter
The last few weeks have had several high-profile incidents in which expert book authors were discovered to have badly misread the research they used in their books: vox.com/future-perfect…
Paul Dolan misread the American Time Use Survey as suggesting married women admit they're miserable when their husbands leave the room; actually, 'spouse absent' didn't mean "left the room" but "not living in the home".
Naomi Wolf thought that 'death recorded' in British sentencing documents meant the person had been executed; it was actually a phrase used to commute their sentence.
What's the takeaway here? Well, firstly, anyone can publish a book. Books are not peer-reviewed. Books are not carefully fact-checked. You should not believe surprising factual claims in a book unless you already trust the author.
Secondly, it's really easy to misinterpret survey data. When you see a summary that links to an official-looking source, read the actual source, or it's fairly likely that crucial information will have been lost in translation.
Thirdly, in these cases the data was at least public, so other people could dig into it and find the truth. In many cases, research is based on proprietary data no one else can verify. We have to stop putting up with that. Without external verification, the error rate is absurd.
So 1) don't trust books 2) don't trust summaries or analyses without checking the source 3) don't trust anything published without the data and code. That's actually a lot of skepticism to apply to everything you read. It's kinda rough. But you want to believe true things, right?
