Your head is there to move you around: Goal-driven models of the primate dorsal pathway
Preprint
Abstract
Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representations increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contrast, there are no image-computable models of the dorsal stream with comparable explanatory power. We hypothesized that the properties of dorsal stream neurons could be explained by a simple learning objective: the need for an organism to orient itself during self-motion. To test this hypothesis, we trained a 3D ResNet to predict an agent’s self-motion parameters from visual stimuli in a simulated environment. We found that the responses in this network accounted well for the selectivity of neurons in a large database of single-neuron recordings from the dorsal visual stream of non-human primates. In contrast, ANNs trained on an action recognition dataset through supervised or self-supervised learning could not explain responses in the dorsal stream, despite also being trained on naturalistic videos with moving objects. These results demonstrate that an ecologically relevant cost function can account for dorsal stream properties in the primate brain.Keywords
This publication has 64 references indexed in Scilit:
- Hierarchical processing of complex motion along the primate dorsal visual pathwayProceedings of the National Academy of Sciences, 2012
- A Three-Dimensional Spatiotemporal Receptive Field Model Explains Responses of Area MT Neurons to Naturalistic MoviesThe Journal of Neuroscience, 2011
- Selectivity and Tolerance (“Invariance”) Both Increase as Visual Information Propagates from Cortical Area V4 to ITThe Journal of Neuroscience, 2010
- Decoding of MSTd Population Activity Accounts for Variations in the Precision of Heading PerceptionNeuron, 2010
- Learning receptive fields using predictive feedbackJournal of Physiology-Paris, 2006
- Spatiotemporal Structure of Nonlinear Subunits in Macaque Visual CortexJournal of Neuroscience, 2006
- Underlying principles of visual shape selectivity in posterior inferotemporal cortexNature Neuroscience, 2004
- Spike-triggered characterization of excitatory and suppressive stimulus dimensions in monkey V1Neurocomputing, 2004
- Temporal Precision of Spike Trains in Extrastriate Cortex of the Behaving Macaque MonkeyNeural Computation, 1996
- Model for the computation of self-motion in biological systemsJournal of the Optical Society of America A, 1992