Thread by @davisblalock

"Language Models Can Teach Themselves to Program Better"

This paper changed my thinking about what future langauge models will be good at, mostly in a really concerning way. Let's start with some context: [1/11]

To teach models to program, you used to give them a natural language prompt. But recent work has shown that you can instead just show them a unit test and tell them to… [2/11]

…generate a program that satisfies it (a “programming puzzle”). This is way nicer because it’s simpler and you can just run the code to see if it works. [3/11]

What this paper proposes to do is use a language model to generate such puzzles, and then use the puzzles to further train the model. You can check whether the puzzles are valid programmatically, so this gives you a scalable way to generate training data. [4/11]

This approach works really well. They find that using the generated (puzzle, solution) pairs helps a lot, as does only using pairs where the solution is correct. [5/11]

This is strong research with a clever method and cool results, but the implications worry me. The Chinchilla paper (arxiv.org/abs/2203.15556) shows that training data is currently the limiting factor in… [6/11]

…scaling up language models. But if we can generate unlimited training data for coding in particular, this suggests that *future language models will be far better at coding than anything else.* [7/11]

This might be bad because coding ability is the main thing that could make deployed models hard to control—like, no amount of English proficiency lets you modify… [8/11]

…your own source code or break out of a Docker container. But if you can code at a superhuman level? Things could get interesting... [9/11]

Also, I know some people think teaching AI to code is great, while others think that misaligned AI is the #1 threat to humanity (e.g., @ESYudkowsky and company), so hopefully this paper spurs some good discussion. [10/11]

@ESYudkowsky Paper: arxiv.org/abs/2207.14502

If you like this paper, consider RTing this (or another!) thread to publicize the authors' work, or following @adamfungi

For more paper summaries, you might like following @mosaicml, me, or my newsletter: bit.ly/3OXJbDs [11/11]

https://twitter.com/davisblalock/status/1558347542101839873