Thread by @mrcslws

New in @FrontiersIn: "Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells" frontiersin.org/articles/10.33…

Here on Twitter, I'll unbundle 3 interesting ideas that the paper is built on.

Idea 1: Location representations can be specific to objects. Grid cells (in their idealized form, with multiple modules) demonstrate one way a single population of neurons could represent location and object identity simultaneously.

Computationally, this is nice because the system doesn't need to coordinate activity of separate "location" and "object identity" populations. When multiple locations-on-objects are active at the same time, the system doesn't confuse which location goes with which object.

Idea 2: Sensory input can be encoded as a "location in an external reference frame". Given the location and orientation of your "sensor" (your eyes, your finger, a camera, etc.) in the reference frame of the sensed object, you can predict the sensor's input.

Locations are a nice way to represent sensory input. Given motor input, a location can be updated, predicting the next sensory input. Or, given new sensory input, the displacement between the two locations indicates movement information (suggesting a mechanism for optic flow).

Idea 3: Object recognition can be framed as a localization problem. Normally, "localization" refers to recognizing your environment and your location in that environment. Here we approach object recognition as a similar problem, but with "objects" and "sensors".

These three ideas aren't entirely dependent — a variation of this model could work without Idea 1 — but they fit together quite nicely.