Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Hyung Won Chung Profile picture
Hyung Won Chung
@hwchung27
Research Scientist @OpenAI. Past: @Google Brain / PhD @MIT
Sep 19, 2024 • 11 tweets • 3 min read
Here is my talk at @MIT (after some delay😅)

I made this talk last year when I was thinking about a paradigm shift. This delayed posting is timely as we just released o1, which I believe is a new paradigm.

It's a good time to zoom out for high level thinking.

(1/11) Full video:

Slides:

Here is the talk summary 🧵

(2/11)
docs.google.com/presentation/d…
May 17, 2023 • 7 tweets • 3 min read
I gave an invited lecture on Instruction finetuning and RLHF for @hhexiy 's class at NYU.

One unique perspective of my lecture is that I introduce RLHF as an instance of using a learned objective function.

Video:
Slides: docs.google.com/presentation/d… Image 1/ I start with the instruction finetuning and discuss 2 flavors that use
a) academic datasets (e.g. Flan)
b) the "API" data

The distinction is important because a) have long input and short target.

Meanwhile we are increasingly interested behaviors with long form generation. Image
May 16, 2023 • 4 tweets • 1 min read
In 2013, I took a class taught by Prof Strang. At the time he had been teaching at MIT for 52 years.

He continued teaching for another 10 years. Yesterday he gave his final lecture youtube.com/live/lUUte2o2S…

He taught me how to like Linear Algebra and how invaluable teaching is. A few years later, I took a machine learning class which Prof. Strang audited. Once he was standing at the aisle of the classroom (26-100).

It was so inspiring to have him as a classmate for an hour 🙂

Fun fact: that classroom happens to be the one he gave his final lecture!
Mar 23, 2023 • 4 tweets • 1 min read
Hey Bard...? 🤔 Image This particular failure is surprisingly reproducible Image
Oct 21, 2022 • 9 tweets • 3 min read
New paper + models!

We extend instruction finetuning by
1. scaling to 540B model
2. scaling to 1.8K finetuning tasks
3. finetuning on chain-of-thought (CoT) data

With these, our Flan-PaLM model achieves a new SoTA of 75.2% on MMLU. We study the scaling phenomena of instruction finetuning and find that scaling both the model size and the number of tasks greatly improves performance.

Key takeaway: we should continue to scale instruction finetuning!

(though gains were slightly smaller after 282 tasks)