Thread by @sfiscience

"Humans are prone to giving machines ambiguous or mistaken instructions, and we want them to do what we mean, not what we say. To solve this problem we must find ways to align AI with human preferences, goals & values."
- @MelMitchell1 at @QuantaMagazine:
quantamagazine.org/what-does-it-m…

“All that is needed to assure catastrophe is a highly competent machine combined with humans who have an imperfect ability to specify human preferences completely and correctly.”

- Stuart Russell (@UCBerkeley) as quoted by @MelMitchell1 in her latest @QuantaMagazine article

"It’s a familiar trope in #ScienceFiction — humanity threatened by out-of-control machines who have misinterpreted human desires. Now a not-insubstantial segment of the #AI research community is concerned about this kind of scenario playing out in real life."
- @MelMitchell1

"People are often irrational and behave in ways that contradict their values, and values can change over individual lifetimes and generations...it’s not clear whose values we should have machines try to learn."
- @MelMitchell1
on #AIAlignment for @QuantaMagazine

"An essential first step toward teaching machines ethical concepts is to enable machines to grasp humanlike concepts in the first place, which I have argued is still #AI’s most important open problem."
- SFI Prof @MelMitchell1 on "the barrier of #meaning":
melaniemitchell.me/PapersContent/…

"Without a better understanding of what #intelligence is and how separable it is from other aspects of our lives, we cannot even define the problem, much less find a solution. [It] won’t be easy; it will require us to develop a broad, scientifically based theory of intelligence."