I like data. I think having data is vital to addressing social problems. But a graph of maternal deaths that circulated on Twitter this week shows the limits of trusting numbers too much. Precision creates an illusion of accuracy and that can be dangerous. 1/12 
Data can show how trends are behaving over time and create comparisons across geographies that lead to learning and best practices. This maternal death data looks terrible for the US. It shows rising deaths and terrible comparisons to other 1st world countries. 2/12
The ? is whether the data accurately represent what’s happening and the answer is a clear no. @Propublica wrote great article in '17 detailing problems with this data & concluded "what data exists on [maternal deaths] is incomplete and untrustworthy." 3/
propublica.org/article/how-ma…
propublica.org/article/how-ma…
@propublica Starting in 2003, new standards led to a checkbox being added to death certificates that asked about pregnancy. States that weren’t already collecting that info added the checkbox sporadically over the next 14 years. 4/12
@propublica What was previously a complicated, deductive task of figuring out the underlying cause of death for a national health agency employee became an easy task of counting check marks. What do you think happened? 5/12
@propublica Soon after a state added the box, the # of reported maternal deaths rose dramatically. “Adding the checkbox almost doubles a state’s reported maternal mortality rate,” a researcher told ProPublica.
Maternal health didn't change; how we counted maternal health changed. 6/12
Maternal health didn't change; how we counted maternal health changed. 6/12
@propublica This makes comparisons across time meaningless. And given that states counted maternal deaths in different ways at different times, it's hard to derive much from geographic comps. But people see the data without context and (justifiably) assume there's an exploding crisis. 7/12
@propublica Yes, America’s numbers look terrible relative to the rest of the world, but we don’t know if that’s truth or also just a counting issue. If we can’t trust data from US hospitals, should we trust data from rural hospitals in Spain? I don’t. 8/12
@propublica We should always strive for better maternal health, but this data gives an erroneous view of what’s happening. We don’t know the underlying trend or where the US ranks relative to our peers. Yet we use this data and others like it to make policy and set priorities. 9/12
@propublica This makes it hard to know what is working, what is best practice, and whether this is a crisis. This is not unique to maternal deaths, or healthcare. This recent article shows the difficulty in relying on official crime statistics. 10/12 newyorker.com/culture/annals…
@propublica Data exists for female suicide in Venezuela (1.2/100,000), robberies in Ecuador (398.8/100,000), hectares planted to marijuana in Albania (219,000), and alcohol addiction in Rwanda (17.4%). All very precise numbers. But do you believe any of it? 11/12
@propublica The lesson here is don’t just regurgitate numbers without investigating the integrity of the data source. Even when the source has no reason to be biased, it doesn’t mean the data is accurate. “Lies, damned lies, and statistics” is too often the truth. 12/12
