Linking connectivity, dynamics and computations in recurrent neural networks
Authors:
Preprint
Abstract
Large scale recordings of neural activity in behaving animals have established that the transformation of sensory stimuli into motor outputs relies on low-dimensional dynamics at the population level, while individual neurons generally exhibit complex, mixed selectivity. Understanding how low-dimensional computations on mixed, distributed representations emerge from the structure of the recurrent connectivity and inputs to cortical networks is a major challenge. Classical models of recurrent networks fall in two extremes: on one hand balanced networks are based on fully random connectivity and generate high-dimensional spontaneous activity, while on the other hand strongly structured, clustered networks lead to low-dimensional dynamics and ad-hoc computations but rely on pure selectivity. A number of functional approaches for training recurrent networks however suggest that a specific type of minimal connectivity structure is sufficient to implement a large range of computations. Starting from this observation, here we study a new class of recurrent network models in which the connectivity consists of a combination of a random part and a minimal, low dimensional structure. We show that in such low-rank recurrent networks, the dynamics are low-dimensional and can be directly inferred from connectivity using a geometrical approach. We exploit this understanding to determine minimal connectivity structures required to implement specific computations. We find that the dynamical range and computational capacity of a network quickly increases with the dimensionality of the structure in the connectivity, so that a rank-two structure is already sufficient to implement a complex behavioral task such as context-dependent decision-making.Keywords
All Related Versions
This publication has 69 references indexed in Scilit:
- Once upon a (slow) time in the land of recurrent neuronal networks…Current Opinion in Neurobiology, 2017
- Recurrent neural networks as versatile tools of neuroscience researchCurrent Opinion in Neurobiology, 2017
- On simplicity and complexity in the brave new world of large-scale neuroscienceCurrent Opinion in Neurobiology, 2015
- Neural circuits as computational dynamical systemsCurrent Opinion in Neurobiology, 2014
- Outliers in the spectrum of iid matrices with bounded rank perturbationsProbability Theory and Related Fields, 2011
- Decoding a Perceptual Decision Process across CortexNeuron, 2010
- Generating Coherent Patterns of Activity from Chaotic Neural NetworksNeuron, 2009
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- Large deviations and mean-field theory for asymmetric random recurrent neural networksProbability Theory and Related Fields, 2002
- Noise, neural codes and cortical organizationCurrent Opinion in Neurobiology, 1994