I would normally never send anyone to Lesswrong. com, but someone posted about Sam Altman remarks about OpenAI’s plans for GPT-4, and I have thoughts lesswrong.com/posts/aihztgJr… 1/7
GPT-4 will focus on coding (ala Codex). It will not be much bigger than GPT-3. The focus will instead be "line of sight" planning. Which is not really planning, it just means bigger context windows and output windows. 2/7
A long enough output window looks like lookahead, but is still just applying historical patterns to new problems. Dare I say "case based reasoning"? 3/7
"GPT-5 might be able to pass the Turing test. But probably not worth the effort." Perhaps, but only because the Turing Test is a really bad test. Anyway, chatbots have been "beating" the Turing Test for the last 10 years. It's not important. 4/7
Other notes: focus on making language models tell the truth and avoid "out of bound topics". This is the controllability problem. Still a problem. I have a lot of thoughts on how to improve controllability, too many to share right now. 5/7
DALL-E will be publicly released. 6/7
A bunch of AGI mumbo-jumbo, but one real concession: AGI will likely require new algorithmic breakthroughs that haven't happened yet, instead of just building bigger models. This is my belief: we don't yet have a workable theory on how to reach AGI. 7/7

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Mark Loki Variant Riedl

Mark Loki Variant Riedl Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @mark_riedl

4 Sep
I used GPT-J to create new loot items Image
Sometimes GPT gets a bit over-excited and starts to tell a story instead Image
It took longer to download and launch GPT than it took me to engineer an prompt that worked colab.research.google.com/drive/1Y392okU…
Read 5 tweets
1 Aug
In about 3 weeks universities will be in session again. Many universities (like my own) want to pretend that things will be back to normal. The buildings and classrooms and quads will all be there and look the same. The routines of commuting to classes will be the same… 1/7
But WE will not be the same. We may still be suffering from mental fatigue. We may have developed new life routines and work habits that are suddenly incompatible with on-campus life. 2/7
2nd year students will be expected to act like 2nd year students even though they are navigating the new social norms of being away from home for the first time—something students normally learn in their first years. 3/7
Read 7 tweets
17 Apr
For some insane reason, my team submitted 7 papers to the NAACL Workshop on Narrative Understanding.

Even more insane: all seven were accepted!
1. Fabula Entropy Indexing: Objective Measures of Story Coherence
@lcastricato @spencerfrazier @JonathanBalloch

A new way to OBJECTIVELY measure the coherence of story generation systems. Grounded in narratology and validated in controlled studies
2. Towards a Formal Model of Narratives
@lcastricato @recardona @DavidThue

A narratological theory that makes narratives mathematical. This is what makes the above evaluation method work. In other words, a practical theory.

Read 9 tweets
15 May 20
I’m finally ready to release my neural net based lyrics parody generation system…

Introducing: Weird A.I. Yankovic!

Runs on Google Colab: colab.research.google.com/drive/12g07FS2…
You can provide the rhyme scheme and syllables per line for an existing song, and it will write new lyrics to match.

In the true spirit of parody, here is a Michael Jackson song (“Beat It”) rewritten to be about food.

Then you can sing the song yourself to the horror of others
With a little bit of extra work (you provide a mp3 or mp4), the system will produce a karaoke video to make it easier to sing along
Read 10 tweets
21 Sep 19
Hollywood: It’s about an experimental AI that—

Me: that fails to converge until the programmer cleans up millions of lines of labeled data? Right! Very suspenseful! Never know if that’s going to work.

Hollywood: you’re fired
Hollywood: the AI has a robot body that—

Me: can’t pick up any objects unless they a placed in a very specific way on a table at just the right height. The struggle is real.

Hollywood: No—
Read 7 tweets
19 Feb 19
This is a problem for the DOD and has been for a long time wired.com/story/the-pent…

But it is addressable by fixing our higher education system, which is not producing enough AI/ML engineers and skewed toward a small # of elite universities.
Under the belief that there are no secret algorithms, only secret engineering, the DOD mostly needs people that can build hardened AI/ML systems (let’s ignore the question of what these systems are for for the moment).
Right now there is high demand in industry for these skills. The demand is high because when it comes to AI and ML, our education system is skewed toward a small number of elite universities that have the resources to do research and thus train graduate students in these areas.
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!