Approximating Nonlinear Functions With Latent Boundaries in Low-Rank Excitatory-Inhibitory Spiking Networks
Open Access
- 23 April 2024
- journal article
- research article
- Published by MIT Press in Neural Computation
- Vol. 36 (5) , 803-857
Abstract
Deep feedforward and recurrent neural networks have become successful functional models of the brain, but they neglect obvious biological details such as spikes and Dale’s law. Here we argue that these details are crucial in order to understand how real neural circuits operate. Towards this aim, we put forth a new framework for spike-based computation in low-rank excitatory-inhibitory spiking networks. By considering populations with rank-1 connectivity, we cast each neuron’s spiking threshold as a boundary in a low-dimensional input-output space. We then show how the combined thresholds of a population of inhibitory neurons form a stable boundary in this space, and those of a population of excitatory neurons form an unstable boundary. Combining the two boundaries results in a rank-2 excitatory-inhibitory (EI) network with inhibition-stabilized dynamics at the intersection of the two boundaries. The computation of the resulting networks can be understood as the difference of two convex functions and is thereby capable of approximating arbitrary non-linear input-output mappings. We demonstrate several properties of these networks, including noise suppression and amplification, irregular activity and synaptic balance, as well as how they relate to rate network dynamics in the limit that the boundary becomes soft. Finally, while our work focuses on small networks (5-50 neurons), we discuss potential avenues for scaling up to much larger networks. Overall, our work proposes a new perspective on spiking networks that may serve as a starting point for a mechanistic understanding of biological spike-based computation.Keywords
All Related Versions
This publication has 89 references indexed in Scilit:
- Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activityCurrent Opinion in Neurobiology, 2021
- Decoding and encoding (de)mixed population responsesCurrent Opinion in Neurobiology, 2019
- Neuroscience out of control: control-theoretic perspectives on neural circuit dynamicsCurrent Opinion in Neurobiology, 2019
- Towards the neural population doctrineCurrent Opinion in Neurobiology, 2019
- Recurrent neural networks as versatile tools of neuroscience researchCurrent Opinion in Neurobiology, 2017
- Why neurons mix: high dimensionality for higher cognitionCurrent Opinion in Neurobiology, 2016
- Variations and extension of the convex–concave procedureOptimization and Engineering, 2015
- Neural circuits as computational dynamical systemsCurrent Opinion in Neurobiology, 2014
- The use and abuse of large-scale brain modelsCurrent Opinion in Neurobiology, 2014
- On the expressibility of piecewise-linear continuous functions as the difference of two piecewise-linear convex functions1986