🧵Let's talk about the new #bot study by @BrownUniversity researchers that makes the rounds in media with scaremongering claims.
@sciam / @guardian have uncritically reproduced statements from the authors even though we should be extremely skeptical. Let me explain why: 1/7
@sciam / @guardian have uncritically reproduced statements from the authors even though we should be extremely skeptical. Let me explain why: 1/7
Straight to the point: without having access to the user ids of the classified “bots” in their study we should question any claim they make. We (@JonasKaiser ) explain this our paper (journals.plos.org/plosone/articl…) about the false-positive problem of their approach. 2/7
Their approach is even questioned by the creators of the detection tool (@Botometer)👇. We show in our own paper that with thresholds (also the one used in the Brown study) researchers will end up with a high number of false-positives and thus overestimate the number of bots. 3/7
Researchers working with these tools (not the creators!) are usually unwilling to share the ids of as “bots” classified accounts even though Twitter's TOS allow this. There is only one case in which researchers were willing to share the ids and the validation was devastating: 4/7
A manual validation of a random sample by @FlorianGallwitz showed that out of 130 as “bot” classified accounts none is probably a social bot. I expect the same with this study. It's also a possible explanation why researchers are reluctant to share ids 5/7
https://twitter.com/JonasKaiser/status/1321449516705501188
I have written about these issues before: Why journalists should stop reporting uncritically about bot studies:
https://twitter.com/OuzhouAdi/status/12637614497022238726/7
Scary headlines produced by university PR was one main motivation for my critical analysis of bot classification tools with @JonasKaiser 7/7
• • •
Missing some Tweet in this thread? You can try to
force a refresh
