New paper from my group:

"Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks". 🧵 (1/9)

arxiv.org/abs/2311.09247 We compare performance of people, GPT-4 (text-only), & GPT-4 Vision on tasks in the ConceptARC benchmark. (2/9)

I'm reading Yoshua Bengio's new blog post, "How Rogue AIs may Arise".

yoshuabengio.org/2023/05/22/how…

Mostly it's the same arguments as in earlier writings by Bostrom, Russell, etc.

Lots to say about all this but there's one issue I want to point out.

🧵 (1/8) There's something that always strikes me as strange in these arguments.

Bengio frames it as a kind of syllogism:

(2/8)

New preprint from me and Victor Odouard (SFI intern):

"Evaluating Understanding on Conceptual Abstraction Benchmarks"

arxiv.org/abs/2206.14187

🧵below. Understanding abstract concepts has long been an unsolved problem in AI. Recently, there have been many papers purporting to outperform humans on one abstraction domain, Raven's Progressive Matrices.

I have a few comments on an article that appeared in today's Atlantic magazine, written by @StephenMarche: theatlantic.com/technology/arc… 🧵 /1 This is an interesting article on Google's PaLM model. The author reports on his discussions with Google researchers. But I think he reports too credulously on what they say. /2

Very impressive---indeed, awe-inspiring---AI demos this last week, e.g., from OpenAI (image generation) and Google (text generation).

These demos seem to convince many people that current AI is getting closer and closer to human-level intelligence. 🧵

(1/8) But it's worth remembering the Bongard problems, created by a Russian computer scientist in the 1960s as a challenge to AI. These problems require a rich and general understanding of basic concepts such as "same" vs. "different". (2/8)

I enjoyed this talk by @tomgoldsteincs (and recommend it) -- it starts at about 29:00min at players.brightcove.net/679256133001/N…

I do have a few thoughts (1/10)

https://twitter.com/tomgoldsteincs/status/1484609273162309634

I have also written about some reasons for AI Winters (arxiv.org/abs/2104.12871) and have a different perspective from Tom Goldstein (TG). TG hypothesizes AI winters happened because AI tried to go from "zero to Turing Test" by programs that implemented logical reasoning. (2/10)

I read this article by @blaiseaguera with great interest.

It would take a while to give a complete response to Blaise's arguments, but here are some of my (stream of consciousness) thoughts. 🧵

https://twitter.com/blaiseaguera/status/1471615078516465667

This piece makes a lot of interesting arguments. They seem to be in support of two main points. The first one is that it is possible *in principle* to build machines that are intelligent and that understand. I agree, though I feel that we are still quite for from that.

New paper from me: "Abstraction and Analogy-Making in Artificial Intelligence": arxiv.org/abs/2102.10717

🧵 (1/4) This paper is part review, part opinion. I argue that conceptual abstraction is driven by analogy, and that analogy is an understudied area of AI that will be essential to overcoming the brittleness and narrowness of AI systems. (2/4)

@ezraklein I think your statement is a misunderstanding of what GPT-3 is and what it can do. First, GPT-3 is not "getting better" in any way. It was trained once on a huge amount of text data and then its weights were fixed. It does not improve at all as people use it. @ezraklein Second, GPT-3 cannot do "much of we do". It can generate paragraphs of human-sounding text, but it doesn't understand what it "says" in any humanlike way. AI is nowhere near being able to do "much of what we do".

A thought-provoking article by @GaryMarcus . Accordingly, I had some thoughts! Longish thread ahead. /1 wired.com/story/deepmind… First thought: Why frame this as "DeepMind lost $572 million" rather than "Google made a very large, long-term investment in one of its research arms, DeepMind"? /2