Thread by @KevinZollman

For week 1 of my Network Epistemology class we looked at the classic "Wisdom of the Crowds" results. The basic idea is that (sometimes!) groups can be far more intelligent than any of their individual members.

https://twitter.com/KevinZollman/status/1222908218374000643

The first paper is by List and Goodin who look at variations on the classic Condorcet Jury Theorem (CJT).

CJT says that, under the right conditions, when given a choice between two options a group will be far better than any of it members.

onlinelibrary.wiley.com/doi/full/10.11…

List and Goodin look at how different voting rules (like #RankedChoiceVoting) would fare according to the CJT with more than two options. They argue that there isn't much of a difference between different voting rules in this framework, that many of them are very similar.

The class was concerned, however, that this conclusion might depend critically on how List and Goodin modeled "error." And, perhaps, that different models of error might come to different conclusions.

Anybody know about any work on this? I don't.

The second paper was by @JustinWolfers and Zitzewitz on prediction markets. Prediction markets are a different way to aggregate judgments about the likelihood of events that allow people to trade "bets" on them.

aeaweb.org/articles?id=10…

There is tons of work on these markets and the conditions under which they have been successful and unsuccessful. There are many famous successes and failures, these days.

In addition to reliability, economists are interested in situations where the market is arbitrage free. That is, when does the market present an opportunity to make money with no risk -- an arbitrage. (Philosophers know this as a "Dutch book.")

One interesting question raised by the students was: is there an epistemic virtue to being arbitrage free? That is, should we expect better predictions from an arbitrage free prediction market than one which is not?

We had a few ideas, but nothing conclusive.

The last paper is by @ClintinS and colleagues (including my colleague @SteveBroomell). They take on two tasks, building a general model of crowd wisdom and calibrating parts of that model to some already published experiments.

arxiv.org/pdf/1406.7563.…

The model is very interesting and very powerful. I really enjoyed the paper. It allows for agents to be correlated in all sorts of interesting ways, and provides a quantitative way to model what types of correlations are good and what are bad.

Interestingly, they show a version of the "diversity benefit" discussed in some of @Scott_E_Page's work. That is, groups are better when they are negatively correlated with one another. This stands in contrast to the usual discussion of CJT, where people want independence.

The model is super interesting. But it suffers from two limitations. One is that they allow unequal weighting, where one person gets more of a say over the final judgment than others. This is great, when you can do it, but one has to know a lot about individuals to get it right.

Second, like all "pooling" models it doesn't account for the possibility of synergy between agents, where we would come to believe something more strongly because we all agree.

And it can lead to odd consequences like, at the start we all agree that variable X is probabilistically independent from variable Y, but our group judgment renders them probabilistically dependent.

Overall, the first week was really interesting and showed the diversity of methods and results in the "wisdom of the crowds" literature. We are going to come back and explore some of these models from the context of network groups later in the semester.