Thread by @notcomplex_

Does diversity improve a firm's financial performance? Thread 🧵

A new paper was just released criticizing earlier findings. They conclude that previous claims are both empirically and methodologically flawed; diversity has no impact on performance.

McKinsey, a consulting firm, released reports starting in the year 2015 entitled: "Diversity Matters" (2015), "Delivering through Diversity" (2018), "Diversity Wins: How Inclusion Matters" (2020), and "Diversity Matters Even More: The Case for Holistic Impact" (2023).

The previous research (McKinsey) produced a diversity score for different companies and then sought to see differences between the first and fourth quartiles of company diversity. This is their main metric for comparing companies empirically.

Specifically, they claim upwards of a 20% difference in the probability that a company outperformed in their sector if they were in the top diversity quartile as opposed to the bottom.

Here's one of their figures:

This recent paper identified a number of problems with their approach. First, their p-values; it might be hard to see but their first error bars above are just barely above zero.

This is suspicious, if the data analysis is done without any researcher influence then p-values should rarely be just below significance.

In fact, the first report had a p-value of 0.0455 about as close as you can get to non-significance.

Also troubling is the lack of transparency, as the authors write "McKinsey would not provide us with their detailed datasets, nor the names of the firms in their datasets, so we were unable to directly replicate or investigate their analyses."

Closed data is a poor practice and McKinsey seemed less than willing to share anything.

So the researchers replicated McKinsey's method on their own. They went through all 500 members of the SAP 500 to collect diversity metrics and multiple performance measures.

Here's The composition by gender:

And here it is by race:

They also had a breakdown by position (CEO, COO, etc.) the racial composition of all the groups was similar except for the CTO (76% White and 21% Asian) and the Chief Diversify officer (52% White and 38% Black).

They then took their race data and scored int on a 'diversity' metric. This calculates how far off from their representation of the population any executive team is. It's not exactly a well-motivated or robust way of measuring 'diversity' but it's what McKinsey did.

Further, this is not a causal metric, simply measuring diversity and firm performance doesn't reveal what the cause is even if you find a significant result. McKinsey admits this in their reports, so it seems strange that they would entitle this work as 'Diversity Wins'.

The new results are starkly different from McKinsey's. Six (EBIT and five new ones) company performance measures were used and almost everything was not-significant. The only significant value they found using McKinzie's method went in the opposite direction!

Though it's just barely NS so don't put much stake in it.

See the data points for yourself, the regression is essentially a straight line.

But what about causality? The authors did just that in a different paper! They compared the racial composition of firms to their performance in subsequent years. Look to NHHI8 and NHHI5, these are McKinsey's metrics and there are few significant values.

In fact, McKinsey's measure of EBIT found no significant values for all measures.

Also if you take a look at the TBQD metric, one that apportions the diversity score by the ratios of college graduates likely to end up in executive talent pipelines, they find no significant values at all.

Overall this highlights the need for better science, replication, and facts. A non-causal, non-replicable, non-public metric being the cornerstone of one of the top consulting firms' diversity outlook is a very sorry state of things. Let's hope that the research improves.

Sources:
econjwatch.org/File+download/…
dx.doi.org/10.2139/ssrn.4…