Thread by @JeremiahYCohen

Extremely proud of @bari_bilal & colleagues' work:
cell.com/neuron/fulltex…

How does the nervous system maximize reward in a dynamic world? In the brain's panoply of functions, these are among its most wonderful. Theory says that decision policies are updated by feedback.

The link between this feedback (discrepancies between predicted and obtained rewards) and firing rates of dopamine neurons is one of systems neuroscience's success stories. But where are these mental models actually stored? How are they remembered in the time between decisions?

@bari_bilal discovered two variables from the theory, relative value and total value, are represented by long-lasting firing rate changes in medial prefrontal cortex neurons. This was very surprising to see at first; I used to think neurons had "baseline" firing rates.

These neurons' relative-value firing rates were used to bias choices. Total-value firing rates were used to bias vigor (response time). Remarkably, relative-value activity was extremely stable, together with choices themselves (no "forgetting").

In contrast, total-value activity decayed slowly, together with response time! These are exactly the ingredients you would need to tell the rest of the brain what choice it should make and how much energy it should expend.

What is the "rest of the brain" in this case? It turned out that these decision variables were weakly represented in premotor cortex, which is part of the final common pathway for action selection. So which part of the brain listens directly to these variables?

@bari_bilal recorded specifically from neurons that projected to dorsomedial striatum (using antidromic stimulation with collision tests; a beautiful technique, in its ability to yield positive evidence from a lack of response).

Both decision variables made it into dorsomedial striatum, which is thought to combine cortical, thalamic, and neuromodulatory input to balance exploration and exploitation of alternatives. Some puzzles: (1) How do the basal ganglia convert decision variables into actions?

(2) How can a network of cortical neurons maintain persistent activity over tens of seconds, carefully tuning the variables stored in particular neurons? (This balance of robustness and flexibility is amazing.)

(3) How can the nervous system adjust how quickly to learn from feedback, and how to balance exploration with exploitation (perhaps using neuromodulators such as serotonin and norepinephrine)?

Contact me if you're interested in solving any of these puzzles together.

We have received generous support from @HopkinsNeuro, @HopkinsMedicine, @NIH, @MQmentalhealth, @BBRFoundation, Klingenstein-Simons, and Whitehall.