Motor-related signals support localization invariance for stable visual perception
Open Access
- 14 March 2022
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 18 (3) , e1009928
Abstract
Our ability to perceive a stable visual world in the presence of continuous movements of the body, head, and eyes has puzzled researchers in the neuroscience field for a long time. We reformulated this problem in the context of hierarchical convolutional neural networks (CNNs)—whose architectures have been inspired by the hierarchical signal processing of the mammalian visual system—and examined perceptual stability as an optimization process that identifies image-defining features for accurate image classification in the presence of movements. Movement signals, multiplexed with visual inputs along overlapping convolutional layers, aided classification invariance of shifted images by making the classification faster to learn and more robust relative to input noise. Classification invariance was reflected in activity manifolds associated with image categories emerging in late CNN layers and with network units acquiring movement-associated activity modulations as observed experimentally during saccadic eye movements. Our findings provide a computational framework that unifies a multitude of biological observations on perceptual stability under optimality principles for image classification in artificial neural networks.Keywords
Funding Information
- RIKEN BSI and CBS (Institutional grant)
- Japan Society for the Promotion of Science (26290011)
- Japan Society for the Promotion of Science (17H06037)
- Japan Society for the Promotion of Science (C0219129)
- Fujitsu (Collaborative grant)
This publication has 114 references indexed in Scilit:
- How Does the Brain Solve Visual Object Recognition?Neuron, 2012
- Dynamics of Eye-Position Signals in the Dorsal Visual SystemCurrent Biology, 2012
- Visual perception and saccadic eye movementsCurrent Opinion in Neurobiology, 2011
- Imaging retinotopic maps in the human brainVision Research, 2011
- Unsupervised Natural Experience Rapidly Alters Invariant Object Representation in Visual CortexScience, 2008
- Neuronal mechanisms of visual stabilityVision Research, 2008
- Standing Waves and Traveling Waves Distinguish Two Circuits in Visual CortexNeuron, 2007
- Parietal and superior frontal visuospatial maps activated by pointing and saccadesNeuroImage, 2007
- A Model of the Ventral Visual System Based on Temporal Stability and Local MemoryPLOS Biology, 2006
- Separate visual pathways for perception and actionTrends in Neurosciences, 1992