Thread by @FlorianGallwitz

New preprint:

The Rise and Fall of "Social Bot" Research

by @FlorianGallwitz, @MichaelKreil

tl;dr: Studies claiming to investigate "social bots" have, in reality, just investigated artifacts of the flawed detection methods employed. papers.ssrn.com/sol3/papers.cf…

While researchers who use @Botometer usually withhold the userIDs of the accounts they counted as "bots" and thereby avoid verification of their claims, we are grateful that @adamgdunn shared his list of "bots" with us and allowed us to take a closer look.

What we found was no less than devastating for the whole body of @Botometer-based research.

.@Botometer fails miserably in real-life scenarios. Using it to estimate the prevalence of "bots", as is standard procedure in "social bot" research, is impossible even from a theoretical point of view.

We also tried our best to find a "social bot" using the famous Oxford 50-tweet-per-day criterion. (Spoiler: we found real humans.)

The reason why the Oxford approach can't work is simple: "Social bot" researchers fail to take into account human perseverance and dedication.

Unlike most "social bot" researchers, we believe in transparency. We share the lists of accounts we analyzed in the attachments. We share the raw data of our experiments and include links in the paper. If there is any additional data you need, don't hesitate to contact us.

I'll close this thread with the final paragraph of our paper.