Thread by @realscientists

Good morning sci-tweeps. Let’s start off the day by talking about one of my hobbies: the neuroscience of baseball (aka- why is it so darn difficult to hit a baseball?) 1/25

Sometimes I’m accused of being a poor cognitive scientist. Why I don’t study “real” (aka- complex) cognition, like how we play chess or learn a second language or solve math problems? Why study something as simple as whether or not to make an action? 2/25

But if you really take a step back and look at it, deciding to swing at an oncoming pitch is really an impressive computational feat.

So let’s take a moment to consider all the process that go into solving the “simple” problem of swinging at a baseball. 3/25

Let's start a clock the moment the ball leaves the pitcher’s hand. Photons of light bounce off the ball & hit the back of your (the batter’s) eye where the retina sits. Here they are converted to electrical impulses by photoreceptors (rods & cones).
flickr.com/photos/nationa… 4/25

Part of these signals from the eye go to the superior colliculus, which coordinates with regions in your neocortex to detect motion & have your eyes pursue the moving ball. editthis.info/psy3242/File:S…
5/25

The rest of the signals from the retina go to the lateral geniculate nucleus (LGN) of the thalamus. Here motion information (magnocellular pathway) is separated from acuity information (parvocellular pathway). goo.gl/images/JFj25n
6/25

The LGN then sends these signals to the primary visual cortex (V1) that produces the brain’s first “map” of the visual world by breaking what you see into a set of edges and orientations. This is the “lowest” level of conscious vision.

upload.wikimedia.org/wikipedia/comm… 7/25

Now if we stop the clock here at V1, we are at ~100ms. Which means that by the time you are just starting to consciously perceive the ball, you are seeing the world 100ms in the past. 8/25

Let’s restart our clock. The visual signals move from V1 to other early processing regions (extrastriate cortex) that process things like motion. Here they begin to split into two pathways: the dorsal (above) stream & ventral (below) stream.

upload.wikimedia.org/wikipedia/comm… 9/25

The ventral stream (the “what pathway”), which runs along the bottom of the temporal lobe, begins to work on perceiving objects. What is the ball versus what is the pitcher’s face versus what is the bat? 10/25

The dorsal stream (the “where/how pathway”), running along the parietal lobe, processes where things are in space and begin planning making spatially-guided actions (e.g., swinging a bat). This also helps allocate spatial attention to the ball. 11/25

Both visual streams send information to the prefrontal cortex where they meet up with the signals from the superior colliculus. We are now about ~150ms and the brain is at a critical junction of choice. Swing or not? If so, how to swing? 12/25

In order to make this choice, the frontal cortex engages the help subcortical regions. These are known as the cortico-basal-ganglia-thalamic (CBGT) loops and they are critical for making decisions in the face of uncertainty. 13/25

Our best guess is that the CBGT loops evaluate all possible actions and, using competition between pathways that want to engage an action and those that want to suppress the same action, they bias the cortex towards one decision over another. 14/25

The CBGT are tuned over time through reinforcement signals by the neurochemical dopamine. When an action is rewarded (e.g. home run), it increases the likelihood it is selected in the future and vice versa if it is not rewarded (e.g., a strike).

frontiersin.org/files/Articles… 15/25

After a few dozen milliseconds the CBGT pathways facilitate a specific choice (e.g., swing low) that the prefrontal cortex sends to the premotor cortex that begins to plan the precise sequence of muscle movements needed to execute the movement. 16/25

This “motor plan” then gets relayed to the primary motor cortex whose cells project directly to the spine to control the muscles (along the way they also send information to an “emergency brake” in the CBGT pathways and the cerebellum). 17/25

Our best estimate is that it can take 80-100ms for signals from the primary motor cortex to reach the muscles via the spine. If you are keeping track, we are just starting to engage the muscles about 200ms after the first light from the ball hit the eye. 18/25

Now as your muscles begin to move the brain (and spine) need to keep a close eye on balancing which muscles contract and which relax, with millisecond timing. Any slight error could mean a bad movement. You might swing too high, too late, or with the wrong amount of force. 19/25

But remember, your brain is living in the past because of the lag processing the visual signals. How does it avoid swinging too late? Well along the way the brain predicts where it thinks the ball will be ~200ms in the future & works off of those predictions instead. 20/25

This prediction ends up being what you actually “see.” Which means your mind’s eye fools you into believing your brain’s best prediction of the world, not what your eye’s actually see. This is easily demonstrated by the “flash-lag illusion”.

21/25

This sort of prediction happens for a lot of your senses, including the sense of your body in space (i.e. proprioception), in order to get past the sensory lags.

So how does the brain know when its predictions are accurate?

The cerebellum.

22/25

The cerebellum monitors the incoming sensory signals with your brain’s predictions of what your senses should be. If they are misaligned, the cerebellum sends out a signal that notifies the rest of the brain that something’s not calibrated correctly and needs to be fixed. 23/25

Thus, if your vision is precise, your decision is correct, your motor plans are efficient, & the sensory predictions accurate enough, you might just be able to hit the ball. This requires the collective effort of billions of neurons, across hundreds of different brain areas 24/25

Now here is the real kicker.

All of these computations have to happen in less than ½ a second (or just slightly longer than it takes to blink your eye).

Still think action decisions are simple? 25/25