Thread by @Miles_Brundage

Reading the old AI literature is underrated. Will share some fun examples when I’m at my computer later...

One quick example that’s easy to do by phone - “Steps Toward Artificial Intelligence,” Minsky, 1960: courses.csail.mit.edu/6.803/pdf/step…

Remarkable how the taxonomy of research areas roughly resembles today - learning, planning, etc at high level, lower stuff like exploration/hierarchy, etc

Makes you think progress would have been vastly faster if those folks had today’s compute to try all this stuff out.

Obv lots of blind allies but I think they would have pruned way faster in a different universe. He and many others asked all the right Qs.

(Bit of hyperbole as it’s not clear we even have the right Qs today, but point is, most big Qs have been asked for decades)

He did give NLP short thrift though (mostly discussed as a grammar induction prob)

Overall I’d say A+ for being almost 60 years ago

OK, the thread continues now that I can quote from stuff.

As most folks know, AI has gone through many phases. Now deep learning is all the rage, and once expert systems were all the rage. People had the same sorts of discussions about excess hype, AI winters, robustness, etc.

Consider IJCAI 1985, when there was a panel called "Expert Systems: How Far Can They Go?" covering all such topics but in a very different (and in some ways opposite) technological context. See parts 1 and 2 here (will excerpt):
aaai.org/ojs/index.php/…
pdfs.semanticscholar.org/f6aa/427bf112a…

In addition to the meta topics (hype etc.) being discussed, I found an excerpt that illustrates how different the prevailing approach is today... expert systems are all about explicit knowledge and reasoning - neural nets (generally) are abt fast/reactive/"intuitive" processing..

And these different emphases influenced how people characterized what AI could/couldn't do, where it could be applied, etc. Compare these quotes, one recent from @AndrewYNg and one from Stuart Dreyfus in 1985. They are basically the exact opposite of each other.

And to his credit, Dreyfus was very clear that he wasn't saying intuition was forever unsolvable, and even plugged connectionism as a way to get at it.

Last throwback of the night - there were really fascinating arguments back in the day about why commonsense reasoning was so hard, and how to solve it. One approach championed by Feigenbaum and Lenat was to encode a lot of world knowledge by hand. ...

People like mentioning Cyc as a failure, but I find it more interesting to look at the detailed arguments people made for why it made sense, especially as people are thinking about ML-augmented approaches to address the same problem today. This is my favorite example -

In "On the Thresholds of Knowledge," Lenat and Feigenbaum (1987) laid out in detail one line of thinking for how AI should be solved, and present several hypothesis related to the relationship between knowledge, learning speed, etc. including this little number here:

I generally recommend poking around in AI paper/conference history (Quest for AI, Machines Who Think, Artificial Dreams, and other books are good, too. Other threads to follow are @rodneyabrooks /@etzioni et al. debates in the 80s/90s +anything John McCarthy-related. /Fin

P.S. forgot the link to Lenat + Feigenbaum - here it is: ijcai.org/Proceedings/87…

OK I couldn't resist, one more: lots of funny stuff/notes to self in McCarthy's unfinished book including this to-be-maybe-deleted jab at ML. jmc.stanford.edu/articles/logic…