Thread by @jonathanstray

THREAD on a wacky problem that AI ethics theorists should be aware of.

Consider a coin flip bet: Heads you get 1.5 times your current wealth. Tails you get 0.6. Should you take it?

Expected value is 1/2*0.6 + 1/2*1.5 = 1.05 times your wealth. So take it right?

Not quite...

Averaging over many people, it's true that the expected value of wealth increases. But if you watch one person's wealth over time, it decreases!

If you work it out, you get sqrt(0.6*1.5) ~= 0.95 times as much wealth at each time step.

(from ergodicityeconomics.com/lecture-notes/)

How can this be? Turns out, a few people get very rich taking this bet repeatedly, while most go broke.

In other words, the classic "expected value" framework has a built-in hidden assumption about how to count the outcomes of multiple people.

If you've studied utilitarian ethics, you know that you can't get the utility of an entire society just by adding up the utilities of individual people.

There are entire books on this "aggregation" issue in welfare economics.

It comes down to the ethics of inequality.

And yet, the single most common tool for probabilistic decision making -- the expected value -- is an (infinite) average over possible outcomes. If you ask everyone in a society to decide this way, you are implicitly adding their utilities.

This issue is so severe that there is a major research project underway to restructure economics, and the entire concept of a rational agent, without expected values.

And AI ethics research shouldn't use it uncritically either.
aeon.co/amp/ideas/how-…

This thread brought to you by "TMI for Twitter" and "you should really go write a paper."

Oh and if you're coming from finance and turned off by talking about inequality, here's Taleb talking about the same problem medium.com/incerto/the-lo…