Thread by @jayelmnop

Excited to present 3 #NeurIPS2022 papers on a trend I've been very excited about recently: blurring the boundaries between language models and RL agents

(+a bonus 4th paper on active learning!)

🧵(0/7)

PS: I'm on the industry job market!

1️⃣ Improving Intrinsic Exploration with Language Abstractions

Using language abstractions to guide exploration in RL, e.g. by self-designing a curriculum of increasingly difficult language goals

https://twitter.com/jayelmnop/status/1496515121400922113

Also see @ykilcher review:

(1/7)

2️⃣ Improving Policy Learning with Language Dynamics Distillation (led by @hllo_wrld)

Increasing RL sample efficiency by pretraining agents to model env dynamics from language-annotated demonstrations

https://twitter.com/hllo_wrld/status/1580342726041616385

(2/7)

3️⃣ STaR: Bootstrapping Reasoning with Reasoning (led by @ericzelikman, @Yuhu_ai_)

Improving multistep reasoning in LMs by bootstrapping off of self-generated rationales

Essentially doing RL in chain-of-thought rationale space!

https://twitter.com/Yuhu_ai_/status/1508953753143156743

(3/7)

4️⃣ (bonus!) Active Learning Helps Pretrained Models Learn the Intended Task (led by @AlexTamkin)

Revisiting classic active learning techniques in the context of modern foundation models and few-shot task ambiguity

https://twitter.com/AlexTamkin/status/1516632180532731904

(4/7)

To recap, I see LMs and RL converging from 2 directions:

RL➡️?⬅️LMs

Starting from RL: imbuing agents w/ language priors [1️⃣,2️⃣]
Starting from LMs: improving reasoning not from static corpora, but RL exploration & interaction [3️⃣]

Excited for these paths to intertwine!

(5/7)

I'm on the job market! Mostly industry (+startups). Interested in both traditional RS positions *and* applied roles deploying products to users and improving from feedback. Please DM or reach out at NeurIPS!

(Also reach out in general, happy to chat about anything)

(6/7)

Thank you to my wonderful coauthors: @robertarail, @hllo_wrld, @MinqiJiang, @_rockt, @egrefen, @LukeZettlemoyer, @ericzelikman, @Yuhu_ai_, @AlexTamkin, Salil Deshpande, Dat Nguyen, and Noah Goodman!

(7/7)