Thread by @DanHendrycks

As AI systems become more useful, people will delegate greater authority to them across more tasks.
AIs are evolving in an increasingly frenzied and uncontrolled manner. This carries risks as natural selection favors AIs over humans.

Paper: arxiv.org/abs/2303.16200 (🧵 below)

Other AI scientists have implicitly recognized that this could be an evolutionary struggle and that humans may become the new gorillas.
@geoffreyhinton “There is not a good track record of less intelligent things controlling things of greater intelligence.”

Jürgen Schmidhuber: “In the long run, humans are not going to remain the crown of creation... But that’s okay... you are a tiny part of a much grander scheme which is leading the universe from lower complexity towards higher complexity”

Others like Google's co-founder Larry Page think that “that digital life is the natural and desirable next step in the cosmic evolution”
Page called @elonmusk a “speciesist” for being on the side of humans (which partially caused him to start OpenAI)

@ylecun argues oppositely: “because AI systems did not pass through the crucible of natural selection...[their] intelligence and survival are decoupled, and so intelligence can serve whatever goals we set for it.”
I argue that AIs will in fact be distorted by that crucible.

In the long run, I think AIs can be thought of an invasive species. I discuss ways to mitigate this existential risk in the paper.

Full argument and countermeasures:
arxiv.org/abs/2303.16200
Video explainer:

Paper abstract: