Thread by @mark_riedl

I’ve been told my conversations with the author were influential to this book and that it says nice things about my research

“There’s no manual of human interaction, Riedl sighs”

The topic is a system Brent Harrison and I worked on in 2016, called “Quixote”

Here is one of the papers:
faculty.cc.gatech.edu/~riedl/pubs/ij…

The core idea of Quixote was to teach reinforcement learning systems to follow social behavioral conventions when performing tasks. We introduced “Learning from Stories” as an alternative to “Learning from Demonstrations”.

The work roughly falls into the class of AI alignment.

Quixote had two learning systems. First, it learned a high-level graph, a partial plan (with branches and alternatives), from a crowdsourced corpus of natural language stories.

More here: faculty.cc.gatech.edu/~riedl/pubs/aa…

It learned what the “actions were” and the likely orderings...

The 2nd learning system grounded the high-level actions in the controls of an autonomous (virtual) robot so that it could then learn a RL policy that filled in the missing details and was executable in the environment. B/c natural language stories are not directly executable.

Quixote was highly successful but also really hard to publish. I’m glad it has gotten some recognition.

It launched a number of related research efforts...

Brent Harrison @spencerfrazier & I worked on learning social norms from cartoons: arxiv.org/abs/1912.03553

We used learned norms to make neural language models behave: arxiv.org/abs/2001.08764

We took a crack at teaching altruism to RL agents: arxiv.org/abs/2104.09469