This paper is gold.
I love it. 
Essentially discovering flaws in the Matrix and exploiting them for superpowers.
And this one is just spooky. To win, it found where the answers were stored and Deleted Them. Resulting in a perfect score.
This program won at Tic-Tac-Toe by figuring out how to remotely crash its opponents' computers, causing them to forfeit.
This one hacked a list-sorting task by simply deleting the list so that it was technically no longer unsorted.
Another algorithm discovered that rather than minimizing force, it could apply such a huge force that it overflowed the simulator's memory and registered as zero instead. Of course, the pilot would die, but hey. Perfect score.
More exploitation of flaws in the Matrix: Glitching into the floor would result in a repelling force that could be harnessed for free energy.
The algorithm first had to learn to manipulate time so the glitching was possible.
The algorithm first had to learn to manipulate time so the glitching was possible.
If "kill all humans" is the easiest solution to a problem, then machine learning will do that unless prevented.
But good news! It would have to be the *easiest* solution to the problem. And killing all humans is really hard.
If "bake an unbelievably delicious cake" also solves the problem and is easier than "kill all humans", then machine learning will do *that* instead.
Another example from that paper: Robots had to learn to drive toward a light. Instead of driving straight toward it as expected, they discovered that it worked much better to spin toward the light.
1/2 
1/2 
