Thread by @KevinZollman

It's week 4 in my network epistemology class, and we're looking at the well-known (and controversial) model of group problem solving by Hong and @Scott_E_Page.

This model argues for the benefit of diversity in problem solving.

pnas.org/content/pnas/1…

Before talking about the model, I think it's useful to think about what is the argumentative purpose of the model. My interpretation is that the model is aimed at a particular kind of diversity-skeptic who I'll call "the CEO. "

The CEO says, "I don't care about any of those social justice arguments in favor of affirmative action. I just want to hire the best, and if 'the best' turn out to be all white guys, so be it."

There are many ways to argue with the CEO, and Hong and Page pursue one.

Their argument is, basically, that when one attempts to hire "the best" one induces all sorts of correlations between the members of one's team. Those correlations can be counter productive because you missed an opportunity to bring in someone who is radically different.

Put in economic terms, the marginal contribution of the 8th "best" person is probably smaller than the contribution of someone who is judged by a different metric. That's because the 8th "best" person probably thinks a lot like the first 7 "best" people you already have.

So, even IF you are perfect at picking out the best (and that is a very big IF), you should still favor some diversity over choosing only the best.

Of course, in reality people aren't that good at choosing the best, so that gives an even stronger argument for diversity.

I think it's important to note that this isn't meant at the exclusion of many other positive arguments in favor of diversity. But rather a kind of "worst case" reasoning. Even in the worst case situation for diversity, the CEO should still pay attention to it.

My personal view is that this basic point is extremely interesting and insightful. In addition to discussing that, however, we also wanted to look at the details of the model and criticisms thereof. And there has been a lot of discussion of this particular model.

We spent a long time in the class discussing the particularities of the model and many concerns we had with the assumptions and implementation of the model. I can't possibility reproduce all of them here, but let me highlight a couple.

One interesting criticism was how group dynamics are modeled. There are several possible ways that group dynamics might look different than the model. Some that would favor diversity more, but others that might push back the other way.

In addition, we worried about the ways that the problem space was structured and assumptions about how value was treated by the agents in the model and as a measure for the quality of group output. It's not clear whether altering these assumptions might alter the results.

Beyond our own thoughts, we looked at two papers that are critical of the model. First, is a paper by Thompson.

faculty.cord.edu/andersod/diver…

Thompson's criticism are wide ranging, including correctly identifying and unstated assumption in the PNAS paper -- one that everyone agrees is easily corrected. And the correction strikes me as perfectly fine.

She criticizes other aspects of the model as supporting an exclusive focus on diversity. While she paints this as criticism of the authors, I think it's more fairly directed at some of those who have sloppy applied the model.

She also notes that in a certain sense the model "builds in" the conclusion into the assumptions. For a philosopher, I never quite know how to interpret this kind of criticism. In a sense, it's true of every mathematical theorem -- the conclusion is "contained" in the premises.

But, of course, some theorems are more obvious than others, and so one might wonder how obvious this was. This might just be something that's in the eye of the beholder. Several in the class agreed and felt like the assumptions were too strong and made the result too obvious.

Finally, Thompson expresses a broader concern about how H&P apply mathematics in the social sciences. She points out that the words we apply to mathematical objects in the model are in some sense arbitrary. And the model could be interpreted differently.

This is an interesting point, although quite old now in the philosophical literature. And it is not unique to the social sciences. After all, physicists also have to label what the mathematical quantities in their theory represent as well.

For those interested in following up, there are several interesting response to Thompson's critique. One by Page:

ams.org/journals/notic…

And a longer one by Singer:

journals.uchicago.edu/doi/pdfplus/10…

The last paper we read was by a large collaboration (Grim et al) looking at how Hong and Page define "the best problem solvers" I think it's a great paper which hews close to the H&P model, but explores some particularly interesting assumptions

journals.uchicago.edu/doi/pdfplus/10…

In H&Ps model, hiring the best is equated with "hiring the best for this particular task" which I interpret as motivated by the worst-case-for-diversity reasoning. But, one might be interested if the model's conclusions hold true for other ways of thinking about expertise.

H&Ps models present particular "hard" problems to their agents. H&P also restrict their agents in particular ways that prevent them from being good at many problems at the same time -- you're only "the best" at this one. A student in the class put it, "there are no generalists"

Grim et al. consider one variation, where the problems are made easier (but not too easy). With easier problems, some of H&P's agents become something like generalists. But also, some of H&P's results no longer hold true.

It's a great discussion, which I really enjoyed.

To sum up, modeling creativity -- which is essentially what I take this model to be doing -- is extremely hard. H&P take a shot at it, and their results are suggestive. But there are many ways the model might be made more realistic and I'm excited to see folks take that on.