Brain Decoder Translates Visual Thoughts Into Text
A new generative method called mind captioning can translate visual and recalled thoughts into coherent sentences by decoding semantic patterns across the brain.
Instead of relying on traditional language regions, the system reads structured visual information from widespread cortical areas and aligns it with deep language model features.
This allows it to verbalize what a person sees—or imagines—without requiring internal speech.
The findings suggest that the brain stores richly organized, linguistically expressible information outside the language network, enabling new forms of nonverbal communication.
A new generative method called mind captioning can translate visual and recalled thoughts into coherent sentences by decoding semantic patterns across the brain.
Instead of relying on traditional language regions, the system reads structured visual information from widespread cortical areas and aligns it with deep language model features.
This allows it to verbalize what a person sees—or imagines—without requiring internal speech.
The findings suggest that the brain stores richly organized, linguistically expressible information outside the language network, enabling new forms of nonverbal communication.
Mind captioning translates brain activity into structured sentences—no language network required. The method captures visual and recalled content directly from semantic brain signals, offering a new path for nonverbal communication.
neurosciencenews.com/brain-decoder-…
neurosciencenews.com/brain-decoder-…
Key Facts
Semantic Decoding: The method extracts high‑level semantic information from brain activity and converts it into text through feature alignment with deep language models.
Memory Generalization: It successfully generates descriptions from both viewed and vividly recalled video content, showing cross‑state decoding.
Language Independence: Accurate sentences emerge even when language areas are excluded, revealing that structured visual semantics are encoded outside the classical language network.
Semantic Decoding: The method extracts high‑level semantic information from brain activity and converts it into text through feature alignment with deep language models.
Memory Generalization: It successfully generates descriptions from both viewed and vividly recalled video content, showing cross‑state decoding.
Language Independence: Accurate sentences emerge even when language areas are excluded, revealing that structured visual semantics are encoded outside the classical language network.
“Mind captioning: Evolving descriptive text of mental content from human brain activity” by Tomoyasu Horikawa. Science Advances
doi.org/10.1126/sciadv…
doi.org/10.1126/sciadv…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
