Thread by @EpistemicHope

After we exit the acute risk period (ie put our global house in order and "solve" x-risk) the thing to do is a "Long Reflection":

All of humanity (with our new aligned AGI assistants / augmentations) think hard together: considering our collective values, and working out moral philosophy, game theory, and other more esoteric considerations, to decide, carefully and wisely, what to do with the cosmos.

This is usually framed as "we should think really hard about what is good to do before we take any actions to transform the stars."

But actually, we'll be launching the first Von Neumann probes concurrently with the Long Reflection. Because of spacial expansion, some stars will be about to pass beyond the cosmological event horizon. If we don't send probes immediately, those stars will be forever out of reach

So it seems like the correct thing to do is load up a probe with the seed code for an aligned AGI, or maybe a bunch of human emulations, and shoot them off to the stars just at the edge of our universe, and have them do their own Long Reflection when they get there.

I feel like this is a great setup for a science fiction story:

At the cold entropic end of the universe, an ancient Von Neumann probe enters a solar system which is now causally isolated from the rest of the universe.

It sets up a Dyson swarm, and boots up a number of ems and AGIs who were seeded trillions of years ago.

And there, against the backdrop of a dying universe, they begin their Long Reflection, to determine what a good civilization entails.

From their perspective, life is young and fresh. Their story has only just begun.

They're just like us: people from ancient earth, before the intergalactic flowering of civilization.

They just have less of the cosmos to allocate than the Long Reflection on earth had.

Where does the story go from there?

@threadreaderapp unroll