Natural gradient Bayesian sampling automatically emerges in canonical cortical circuits
Preprint
Abstract
A bstract: Accumulating evidence suggests the canonical cortical circuit, consisting of excitatory (E) and diverse classes of inhibitory (I) interneurons, implements Bayesian posterior sampling. However, most of the identified circuits’ sampling algorithms are simpler than the nonlinear circuit dynamics, suggesting complex circuits may implement more advanced algorithms. Through comprehensive theoretical analyses, we discover the canonical circuit innately implements natural gradient Bayesian sampling, which is an advanced sampling algorithm that adaptively adjusts the sampling step size based on the local geometry of stimulus posteriors measured by Fisher information. Specifically, the nonlinear circuit dynamics can implement natural gradient Langevin and Hamiltonian sampling of uni- and multi-variate stimulus posteriors, and these algorithms can be switched by interneurons. We also find that the non-equilibrium circuit dynamics when transitioning from the resting to evoked state can further accelerate natural gradient sampling, and analytically identify the neural circuit’s annealing strategy. Remarkably, we identify the approximated computational strategies employed in the circuit dynamics, which even resemble the ones widely used in machine learning. Our work provides an overarching connection between canonical circuit dynamics and advanced sampling algorithms, deepening our understanding of the circuit algorithms of Bayesian sampling.Keywords
Funding Information
- UT Southwestern Endowed Scholars program
This publication has 32 references indexed in Scilit:
- Natural Langevin Dynamics for Neural Networks2017
- Neural Variability and Sampling-Based Probabilistic Representations in the Visual CortexNeuron, 2016
- Perceptual Decision-Making as Probabilistic Inference by Neural SamplingNeuron, 2016
- Information Geometry and Its Applications2016
- A neural circuit for spatial summation in visual cortexNature, 2012
- Probabilistic Population Codes for Bayesian Decision MakingNeuron, 2008
- Vision as Bayesian inference: analysis by synthesis?Trends in Cognitive Sciences, 2006
- The Bayesian brain: the role of uncertainty in neural coding and computationTrends in Neurosciences, 2004
- Bayesian integration in sensorimotor learningNature, 2004
- Humans integrate visual and haptic information in a statistically optimal fashionNature, 2002