, 9 tweets, 3 min read Read on Twitter
!!! New paper now out in @NeuronCellPress !!!

We show that neurons in the amygdala and ventral striatum represent different computations when monkeys decide to explore a new opportunity or exploit what they already know

cell.com/neuron/fulltex…
One of the more challenging problems in reinforcement learning is referred to as the explore-exploit dilemma. It refers to the tradeoff we face when we have to decide whether to repeat a past choice or explore a new opportunity.
In noisy environments where you can never be especially certain about the value of anything, we know from seminal studies by @nathanieldaw, @BecketEbs, and others that prefrontal cortex is important for switching between states of exploration and exploitation.
We took a different approach & induced explore-exploit tradeoffs by introducing novel options amidst choice options the monkeys had already learned about.

We know motivational circuits facilitate RL, do they also support computations relevant for deciding when to explore?
The short answer is yes! When we use a partially observable Markov decision process (POMDP) model to break down the value of the monkeys' choices we find different encoding schemes in the amygdala and ventral striatum
When the monkeys choose to explore novelty, the bonus in value associated with exploring and finding out that there is a better option to choose from is more strongly encoded in the amygdala compared to the ventral striatum.
Whereas the immediate expected value of the option, based on the past outcome and choices, is more strongly encoded in the ventral striatum compared to the amygdala.
Therefore at the population level neurons in the amygdala and ventral striatum, simultaneously encode the exploration and exploitation value of choices.
We think this is exciting because it raises some new questions, in a computationally tractable framework, about how motivational neural circuits might contribute to impulsive and compulsive forms of exploratory learning and decision-making.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Vincent Costa
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!