Tweet

Andrew Lampinen

13 Sep, 4 tweets, 2 min read

https://twitter.com/AndrewLampinen/status/1399585478546845698?s=20

I'm pleased to share that we've open-sourced two environments and the hierarchical attention mechanism for our "Towards mental time travel: A hierarchical memory for RL" paper: github.com/deepmind/deepm…

Paper summary thread/links:

https://twitter.com/AndrewLampinen/status/1399585478546845698?s=20

The repository linked above contains 1) a JAX/Haiku implementation of the hierarchical attention module, and 2) an implementation of the Ballet environment, which requires recalling spatio-temporal events, and is surprisingly challenging.

We've also open-sourced the Rapid Word Learning environment, but to simplify dependencies we added these new tasks to the repository for the paper they were based upon (github.com/deepmind/dm_fa…). This environment tests memory extrapolation far out of the training distribution.

We hope these resources will help others to build on our work, and we're excited to hear what you discover! Please reach out if you have any questions or find any bugs—open sourcing required some non-trivial refactoring and it's always possible we missed something.

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @AndrewLampinen

Andrew Lampinen

@AndrewLampinen

1 Jun

How can RL agents recall the past in detail, in order to behave appropriately in the present? In our new preprint "Towards mental time travel: A hierarchical memory for RL agents" (arxiv.org/abs/2105.14039) we propose a memory architecture that steps in this direction.

We draw inspiration from the idea that human memory is like "mental time travel"—we can recall a specific event in the past, and relive it in some sequential detail, with relatively little interference from other events. This ability is key to our goal-directed use of memory.

By contrast, RL agent memories generally lack either this sparsity (attending to one or a few events in the past) or this detail (replaying an event in sequence, rather than just recalling a single vector from each event). This limits their ability to learn from their memory.

Read 10 tweets

Andrew Lampinen

@AndrewLampinen

9 Feb

What are symbols? Where do symbols come from? What behaviors demonstrate the ability to engage with symbols? How do the answers to these questions impact AI research? We argue for a new perspective on these issues our preprint: arxiv.org/abs/2102.03406 Summary in thread: 1/6

We interpret symbols as entities whose meaning is established by convention. We therefore argue that an entity is a symbol *only* to a system that demonstrates active participation in a system of meaning by convention; that is, a system that exhibits symbolic behavior. 2/6

What is symbolic behavior, and how could an AI (or animal) demonstrate it? We outline a set of criteria based on our interpretation of symbols: symbolic behavior is receptive, constructive, embedded, malleable, separable, meaningful, and graded. We illustrate with examples. 3/6

Read 7 tweets

Andrew Lampinen

@AndrewLampinen

12 May 20

How can deep learning models flexibly reuse their knowledge? How can they adapt to new tasks zero-shot, as humans can? In our new preprint (arxiv.org/pdf/2005.04318), we propose a new approach based on learning to transform task representations: meta-mapping. Preview in thread:

Our approach can make drastic adaptations zero-shot, like switching from winning at (simplified) poker to trying to lose. It can allow a visual classification system to recognize new concepts, and can adapt a model-free reinforcement learning to new tasks, without data from them.

It accomplishes this without prior domain knowledge, based only on the relationships between tasks. Specifically, it learns basic task representations, e.g. for poker, via meta-learning. It also learns meta-mappings, higher order tasks which transform these basic task reps.

Read 9 tweets

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Share this page!

Andrew Lampinen

Try unrolling a thread yourself!

More from @AndrewLampinen

Andrew Lampinen

Andrew Lampinen

Andrew Lampinen

Did Thread Reader help you today?

Like this author's thread?