Thread by @karpathy

The hottest new programming language is English

This tweet went wide, thought I'd post some of the recent supporting articles that inspired it.
1/ GPT-3 paper showed that LLMs perform in-context learning, and can be "programmed" inside the prompt with input:output examples to perform diverse tasks arxiv.org/abs/2005.14165

2/ These two [1] arxiv.org/abs/2205.11916 , [2] arxiv.org/abs/2211.01910 are good examples that the prompt can further program the "solution strategy", and with a good enough design of it, a lot more complex multi-step reasoning tasks become possible.

3/ These two articles/papers:
[1] evjang.com/2021/10/23/gen…
[2] arxiv.org/abs/2106.01345
bit more technical but TLDR good prompts include the desired/aspiring performance. GPTs don't "want" to succeed. They want to imitate. You want to succeed, and you have to ask for it.

4/ Building A Virtual Machine inside ChatGPT engraved.blog/building-a-vir…
Here we start getting into specifics of "programming" in English. Take a look at the rules and input/output specifications declared in English, conditioning the GPT into a particular kind of role. Read in full.

5/ "ChatGPT in an iOS Shortcut — Worlds Smartest HomeKit Voice Assistant" matemarschalko.medium.com/chatgpt-in-an-…
This voice assistant is significantly more capable and personalized than your regular Siri/Alexa/etc., and it was programmed in English.

6/ "GPT is all you need for the backend" github.com/TheAppleTucker…
Tired: use an LLM to help you write a backend
Wired: LLM is the backend
Inspiring project from a recent Scale hackathon. The LLM backend takes state as JSON blob and modifies it based on... English description.

7/ The prompt allegedly used by Bing chat, potentially spilled by a prompt injection attack important point for our purposes is that the identity is constructed and programmed in English, by laying out who it is, what it knows/doesn't know, and how to act.

8/ These examples illustrate how prompts 1: matter and 2: are not trivial, and why today it makes sense to be a "prompt engineer" (e.g. @goodside ). I also like to think of this role as a kind of LLM psychologist.

9/ Pulling in one more relevant tweet of mine from a while ago. GPTs run natural language programs by completing the document.

This is not an exhaustive list (people can add more in replies), but at least some of the articles I saw recently that stood out.

It's still early days but this new programming paradigm has the potential to expand the number of programmers to ~1.5B people.