Thread by @AToliasLab

Does the concept of cortical columns extend to higher-level primate cortex?

Using #DeepLearning & physiology, we found that V4 neurons cluster in columns & form functional groups biorxiv.org/content/10.110…

Led by @KonstantinWille @kelli_restivo w/ @sinzlab @kfrankelab @alxecker 🧵

Elucidating the brain’s structure-function organization is key to understanding perception. A classic example are cortical columns, a vertical organization of neurons w/ similar function. They exist in primary sensory areas but it’s unclear if they are present throughout cortex

Here, we asked whether area V4, a mid-level area of the macaque visual system, is organized into functional columns. We presented grayscale natural images & recorded responses of >1200 single V4 neurons in 100 electrophysiological recording sessions using 32-channel depth probes

We fitted the readout of a pre-trained robust ResNet50 on our V4 data to predict neuronal firing rates to natural images, achieving high performance. We treated this model as a “digital twin” of macaque V4 and used it for vast in silico experimentation

We used our model to generate “Maximally Exciting Images” (MEIs) for each neuron. These MEIs exhibited complex features like textures, shapes, or even high-level attributes such as eye-like structures in line w/ tuning properties of V4 neurons identified using parametric stimuli

We verified our MEIs in vivo by developing a closed-loop paradigm for acute electrophysiological recordings of single neurons. The MEI of a single neuron consistently elicited higher neuronal responses than control natural images, suggesting it is indeed the optimal stimulus

We noticed that MEIs from single sessions were more similar to each other than to MEIs from other sessions & quantified this using psychophysics. As neurons from one session are arranged orthogonal to the surface, this suggests that V4 tuning may be organized in a columnar manner

We employed a contrastive learning method to further quantify MEI similarity within and across recording sessions & found that the mean distance between MEIs in the embedding space was significantly smaller within a session than across sessions

Moreover, the selectivity of the neuronal population was clustered in the similarity space, suggesting that V4 tuning properties identified using the neuron’s optimal stimuli can be separated into functional groups selective for a specific feature, reminiscent of cell types

The contrastive learning method was based on this publication: openreview.net/forum?id=nI2Hm… Thanks to @CellTypist @hippopedoid Niklas Bohm for their help in implementing it!

Interestingly, these functional groups closely resemble the feature maps of units in deep neural networks trained on image classification, suggesting that computational principles are shared among biological & artificial visual systems - see work by @ch402 distill.pub/2020/circuits/…

The resemblance between V4 neuronal & deep artificial neural network feature selectivity can be used to generate specific hypotheses about visual tuning properties of primate V4 neurons beyond spatial patterns, such as about color boundary encoding in monkey V4 functional groups

Our findings provide evidence that functional cortical columns may be a generalizable organizing principle in the cortex, beyond primary sensory areas & demonstrates how deep learning can uncover structure-function relationships in the brain by characterizing neuronal tuning

Huge thanks to all other contributors! @ArneNix, @SantiagoACadena, Tori Shinn, Cate Nealley, Gabby Rodriguez, Saumil Patel