Misinformation spreads rapidly online. In response, Facebook & @Twitter have suggested using ordinary users as fact checkers. But our new #OpenAccess article, in the inaugural issue of @journalsafetech, finds this is likely not a viable solution 1/ tsjournal.org/index.php/jots… 
Ordinary users -- and machine learning models based on information from those users -- cannot effectively identify false and misleading news in real time, compared to professional fact checkers, according to our experiment. 2/
In the study, each day 90 Americans and six professional fact checkers (PFCs) evaluated the veracity of a sample of news articles. We collected 12,883 evaluations across 135 articles. 3/
Using this data, we evaluated how well PFCs accurately assessed the veracity of articles -- and compared the results to ordinary users, users with high political knowledge, and machine-learning models based on input from each type of user. 4/
Although machine learning based models perform significantly better than simply aggregating the evaluations of groups of ordinary users, the study found, neither approach is able to perform at the level of PFCs. 5/
In addition, both models perform best when only using the evaluations from those with high political knowledge, suggesting reason for caution for crowdsourcing models that seek to rely on a representative sample of the population See results for simple rules, e.g. 6/
The idea of using ordinary users as fact checkers is appealing. It takes decisions out of the hands of the big, powerful platforms; it is scalable; and it can be more inclusive and representative. Unfortunately, it wasn’t effective in our experiment. 7/
As part of a larger toolkit, real time crowdsourced fact checking may offer a viable tool to combat the spread of online misinformation. Nonetheless we find little evidence that a crowd-based approach, on its own, is sufficient to identify false news. 8/
Congrats to our entire team (@WilliamGodel @zns202 @aslett_kevin @Jonathan_Nagler @RichBonneauNYU @persily @j_a_tucker) for this important research. 9/
To further examine our data, see our open-sourced dataset on Github: github.com/SMAPPNYU/crowd…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
