,
21 tweets,
3 min read
Read on Twitter
(Thread) On artificial general intelligence (AGI), and how we get there:
(1/20) Thought problem: Say you had no knowledge of modern electronics or computer science. Someone hands you an iPad. All you’re allowed to do is tap the screen and watch the display. Think you could understand it well enough to build something that’s functionally equivalent?
(2/20) Do you think it’s possible to capture the deep computational sophistication under the hood of an iPad with a naïve model that is optimized to mimic an iPad’s responses under a broad repertoire of use-cases? Could the model “discover” an underlying instruction set?
(3/20) To me, this seems formally equivalent to the problem of trying to develop algorithms with “general intelligence” using guidance from human behavior alone, and by measuring AGI progress exclusively in terms of performance on difficult tasks that humans perform well.
(4/20) These are valuable metrics of progress, but they can’t be the only metrics. Performing a given task, or a set of tasks, as well as a person doesn’t mean you’ve captured the low-level computations that enable the brain to perform those tasks.
(5/20) Those hidden computational details - the brain’s “instruction set” - are the keys to intelligence. Modern neuroscience is founded on the principle that you can’t understand low-level computations through introspection or by studying behavior alone. You need to dig in.
(6/20) With new tools, we finally have the ability to directly study how neural circuits represent, transform, and learn from data at a scale that’s relevant to understanding behavior. We can develop a circuit theory for the brain. We can understand the instruction set.
(7/20) The foundations of “general intelligence” – capabilities like robust perception, cognition, learning, memory, and adaptive control - are now within reach. In my view, the path lies through neuroscience.
(8/20) Many people feel (or want to believe) that creative engineering approaches coupled with extraordinary compute resources will be sufficient to develop algorithms with general intelligence capabilities. I think that community has a narrow view of general intelligence.
(9/20) My unpopular view: Performing many different tasks (e.g. playing games) well is not an adequate test of general intelligence. (Note: this is not intended to diminish recent achievements in game-playing AI, which are *AMAZING* and will have broad utility.)
(10/20) My critique: I’ve yet to see any major AI research orgs offer a detailed and precise definition of how they define general intelligence. How can anyone have a credible strategy for reaching a goal if they don’t have a precise operational definition of the goal?
(11/20) Neuroscientists fight relentlessly over operational definitions of cognitive processes. But that ultimately makes the field stronger. It gives us a basis for testing and if necessary rejecting hypotheses about what processes are identifiable and dissociable in the brain.
(12/20) The field of AI has fallen into this pattern of assessing algorithms by how they do on benchmark tasks, as a proxy for improved (and often vaguely defined) intelligence capabilities.
(13/20) The AI community measures an algorithm’s “general intelligence” by its performance on multiple tasks, not by a demonstration that it’s possible to identify and dissociate the different computational processes that enable it to do well on those tasks.
(14/20) Creative engineers + more compute is a great recipe for improving algorithmic performance on a broad range of difficult tasks. But you get what you optimize for…
(15/20) Without setting clear and quantifiable targets for underlying intelligence capabilities, I doubt the “creative engineers + more compute” approach will be adequate to get AGI researchers the progress they seek.
(16/20) I’m encouraged by recent community efforts to bridge neuroscience and AI, via the annual Cognitive Computational Neuroscience meeting: ccneuro.org
(17/20) I think the CCN community could add value to AI by defining quantitative performance goals for AGI as a set of specific computational capabilities that neuroscientists already assess in their work, as a complement to current task-based assessments of AI systems.
(18/20) Can we move the field of AI from a task-based understanding of capabilities to a deeper understanding that links low-level computations with the processes they support, and the resulting task performance?
(19/20) In many ways, this is an aspirational goal for neuroscience, as well. But I think there are more people explicitly focused on that goal in neuroscience than in AI, and therein lies the opportunity for cross-pollination of ideas.
(20/20) I’d welcome feedback on the above, which was intended as a constructive critique in the service of accelerating progress in AI. Hopefully others will jump in with their own helpful suggestions for how we should think about the gating challenges for achieving AGI.
