I’ve been told my conversations with the author were influential to this book and that it says nice things about my research
“There’s no manual of human interaction, Riedl sighs”
The topic is a system Brent Harrison and I worked on in 2016, called “Quixote”

Here is one of the papers:
The core idea of Quixote was to teach reinforcement learning systems to follow social behavioral conventions when performing tasks. We introduced “Learning from Stories” as an alternative to “Learning from Demonstrations”.

The work roughly falls into the class of AI alignment.
Quixote had two learning systems. First, it learned a high-level graph, a partial plan (with branches and alternatives), from a crowdsourced corpus of natural language stories.

More here: faculty.cc.gatech.edu/~riedl/pubs/aa…

It learned what the “actions were” and the likely orderings...
The 2nd learning system grounded the high-level actions in the controls of an autonomous (virtual) robot so that it could then learn a RL policy that filled in the missing details and was executable in the environment. B/c natural language stories are not directly executable.
Quixote was highly successful but also really hard to publish. I’m glad it has gotten some recognition.

It launched a number of related research efforts...
Brent Harrison @spencerfrazier & I worked on learning social norms from cartoons: arxiv.org/abs/1912.03553

We used learned norms to make neural language models behave: arxiv.org/abs/2001.08764

We took a crack at teaching altruism to RL agents: arxiv.org/abs/2104.09469

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Mark Riedl is Non-Fungible

Mark Riedl is Non-Fungible Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @mark_riedl

6 Sep 21
I would normally never send anyone to Lesswrong. com, but someone posted about Sam Altman remarks about OpenAI’s plans for GPT-4, and I have thoughts lesswrong.com/posts/aihztgJr… 1/7
GPT-4 will focus on coding (ala Codex). It will not be much bigger than GPT-3. The focus will instead be "line of sight" planning. Which is not really planning, it just means bigger context windows and output windows. 2/7
A long enough output window looks like lookahead, but is still just applying historical patterns to new problems. Dare I say "case based reasoning"? 3/7
Read 7 tweets
4 Sep 21
I used GPT-J to create new loot items Image
Sometimes GPT gets a bit over-excited and starts to tell a story instead Image
It took longer to download and launch GPT than it took me to engineer an prompt that worked colab.research.google.com/drive/1Y392okU…
Read 5 tweets
1 Aug 21
In about 3 weeks universities will be in session again. Many universities (like my own) want to pretend that things will be back to normal. The buildings and classrooms and quads will all be there and look the same. The routines of commuting to classes will be the same… 1/7
But WE will not be the same. We may still be suffering from mental fatigue. We may have developed new life routines and work habits that are suddenly incompatible with on-campus life. 2/7
2nd year students will be expected to act like 2nd year students even though they are navigating the new social norms of being away from home for the first time—something students normally learn in their first years. 3/7
Read 7 tweets
17 Apr 21
For some insane reason, my team submitted 7 papers to the NAACL Workshop on Narrative Understanding.

Even more insane: all seven were accepted!
1. Fabula Entropy Indexing: Objective Measures of Story Coherence
@lcastricato @spencerfrazier @JonathanBalloch

A new way to OBJECTIVELY measure the coherence of story generation systems. Grounded in narratology and validated in controlled studies
2. Towards a Formal Model of Narratives
@lcastricato @recardona @DavidThue

A narratological theory that makes narratives mathematical. This is what makes the above evaluation method work. In other words, a practical theory.

Read 9 tweets
15 May 20
I’m finally ready to release my neural net based lyrics parody generation system…

Introducing: Weird A.I. Yankovic!

Runs on Google Colab: colab.research.google.com/drive/12g07FS2…
You can provide the rhyme scheme and syllables per line for an existing song, and it will write new lyrics to match.

In the true spirit of parody, here is a Michael Jackson song (“Beat It”) rewritten to be about food.

Then you can sing the song yourself to the horror of others
With a little bit of extra work (you provide a mp3 or mp4), the system will produce a karaoke video to make it easier to sing along
Read 10 tweets
21 Sep 19
Hollywood: It’s about an experimental AI that—

Me: that fails to converge until the programmer cleans up millions of lines of labeled data? Right! Very suspenseful! Never know if that’s going to work.

Hollywood: you’re fired
Hollywood: the AI has a robot body that—

Me: can’t pick up any objects unless they a placed in a very specific way on a table at just the right height. The struggle is real.

Hollywood: No—
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!