The degree to which modern AI has given up on trying to understand tractable, thinking systems and just builds them by brute force is interesting. Our understanding of cognition hasn't improved even as we're able to build systems that have near-human performance in some areas.
This has interesting consequences. One is the "money laundering for bias" issue, where machine systems you don't understand pick up on all sorts of latent patterns in the data you feed them. But my favorite is that it removes a favorite fantasy about recursive self-improvement
Transhumanists have been obsessed for years with the nightmare of an AI that starts to recursively self-improve, and in a few hours takes over the world. They take it more seriously than my breezy phrasing of it implies. They have also learned to not talk about it in public
But implicit in this fever dream is the notion that minds are designed, first by their human creators, and then by their owners, iteratively, to be smarter. With the messy kind of AI we have in practice, this feedback loop doesn't work.
It doesn't work for the same reason you can't repeatedly do brain surgery to make yourself better at brain surgery. The brain is a hot goopy mess with everything interconnected. Now our best AI algorithms are becoming like that, too.
So practical AI, instead of the study of cognition, will come to resemble a kind of xenobiology, where we study the behavior of utterly alien, evolved complex systems that for some reason we've deployed at massive scale without thinking through the risks (this is a @zeynep idea)
@zeynep I gave a talk a while back about the dissonance between fantasies of AI (world domination) and the rather anticlimactic status quo (racist linear algebra). It's a really good talk! You should ask Alexa to read it out loud idlewords.com/talks/superint…
@zeynep Also I like to write these talks and you should invite me to give one, though I could do without the complaints that I "ruined the bat mitzvah"
