Thread by @EpistemicHope

I'm increasingly hopeful that the combination of a shift to remote work + the breakdown of the legitimacy of the existing institutions, is a good thing, because power and initiative will accrue to states and cities as they provide notably better governance than their competitors.

It seems to me that moving the locus of power down to a more local level is a win for the world, because it can increase variance in governance.

Because of network effects, a handful of hubs can be disproportionately important to global well-being. Which means that creating highly productive urban centers with unusually good governance waaaay outweighs the cost of all the places that end up with worse governance.

Plus, we would finally, have petri dishes for doing at least some governance innovation.

Gatekeeping on experiments seems to be the main thing that has prevented iterative improvement in large scale institutional mechanism design, which is upstream of so many of our problems.

But if power is reallocated to the city-state level, and those city-states are competing for talent to fuel their economies, we're much closer to having space to try all the ideas that our civilization already has for running things sanely, to see if they actually work and scale.

(Though notably, I'm not sure exactly how that "power being reallocated" thing works. Just because the traditional institutions have lost a lot of legitimacy, the US government still has overwhelming military force.

Is there a gradual, peaceful, path to city-state dominance?)

The Waring States period in China is known for being a flowering of philosophy.

The empire of the Zhou dynasty crumbled, and in the period of chaos, many philosophers and reformers put forward proposals for how to run society better.

It was a violent and confusing time, but also an exciting, and intellectually vibrant time.

With Trump, and covid, and institutions that are obviously sclerotic and clearly lying to us, it feels like the world is getting darker.

But I think maybe we should be pumped for the opportunity. Now, as the old order breaks down, we might be able to build an actually, objectively, better one.

We have the technology.

What we hadn't had, was the opportunity.

That said, I really hope that AGI timelines are long enough that transformative AI arrives AFTER the chaotic period of societal-scale creative destruction and exploration of new modes of organization.

Because chaos is scary, and it seems really bad for AGI to appear on the world stage when most people are scared, watching their world fall away, and desperately trying to hang onto it.

As @VitalikButerin notes here, most human evil is done by people who are afraid.

podofjake.com/2020/08/20/4-v…

And if there's one important consideration that doesn't benefit from increased variance in governance, it's choices about how to deploy technologies that can unilaterally destroy everything.

In that particular case, you have ~ unbounded downside, instead of ~ unbounded upside to many groups trying different approaches.

For most things, we can copy what works so success is approximately a MAX function over all the choices relevant actors make.

But when you're working with things that kill everyone, the most decision maker can game-over everyone.

So success is more like a MIN function.

nickbostrom.com/papers/unilate…

All of which seems bad, since my best (though highly highly uncertain) guess is that we're getting transformative AI sooner rather than later.

But maybe this also gives us the outline of a gameplan:

The foundations of our social order are being undermined. That breakdown might accelerate to a sudden system-wide shock, or it might be gradual.

That's fucking scary.

But it is also a sign that says "go".

Our responsibility is to do the best we can to build and improve on systems that can do better than the ones that collapsed under their own weight.

Take our best ideas and apply them.

Build. Fail. Correct. Repeat.

Copy whats working.

By trial and error, push the world into an equilibrium of sanity-aligned incentives, and good institutional epistemology, and humane values, and...well, adequacy.

Eventually (maybe after a generation or two), things will settle down. People will copy what works. The known-to-be-good institutions and meta-institutions will propagate.

It WON'T be perfect, but civilization will be saner than it has ever been to date.

People will be legitimately hopeful about the future again, because they will be watching their world keep getting better.

And then, on that foundation, of sanity, and cooperation and hope, our AGI clock ticks down to 0, and we FOOM.

. . .

If we have enough time for that.

@threadreaderapp unroll