Reproducible reporting
NYT didn’t fully fact check their front page story.
They just aggregated content from local papers and classified a murder victim as dying of COVID.
Importantly, the public then attempted to reproduce the reporting. And found an error by the 6th name. 🧵
NYT didn’t fully fact check their front page story.
They just aggregated content from local papers and classified a murder victim as dying of COVID.
Importantly, the public then attempted to reproduce the reporting. And found an error by the 6th name. 🧵
Now, this error doesn’t affect the substance of *this* story. Many have indeed died from COVID!
But it does point to an important difference between open source and closed source journalism.
It’s less trustworthy if the public can’t reproduce the reporting.
But it does point to an important difference between open source and closed source journalism.
It’s less trustworthy if the public can’t reproduce the reporting.
It’s analogous to the reproducibility crisis in academia.
Scientists at least publish their methodology & in theory encourage independent replication. Yet many papers don’t replicate.
How much worse is the replication crisis in closed source journalism?
en.wikipedia.org/wiki/Replicati…
Scientists at least publish their methodology & in theory encourage independent replication. Yet many papers don’t replicate.
How much worse is the replication crisis in closed source journalism?
en.wikipedia.org/wiki/Replicati…
Closed source journalism means anon quotes, no links to primary sources, a “trust us” atttiude based on publication’s brand.
Open source journalism means linking to public tweets as quotes, linking to primary sources on (eg) Document Cloud, and allowing public to confirm facts.
Open source journalism means linking to public tweets as quotes, linking to primary sources on (eg) Document Cloud, and allowing public to confirm facts.
Reproducibility and open source would also mean a change in the culture.
Rather than treating corrections as humiliations and marketing NYT as the TRUTH, an alternative is to encourage corrections as pull requests and embrace a model of iterative truth.
Rather than treating corrections as humiliations and marketing NYT as the TRUTH, an alternative is to encourage corrections as pull requests and embrace a model of iterative truth.
