Alexandre Pouget

A bstract: There is a gap between neuroscientific theories of learning and the speed of learning observed in many experiments. Since the Cognitive Revolution of the 1950s, compositionality has played a central role in efforts to bridge this gap. Roughly, a compositional system is one where distinct modules are combined according to a set of rules in order to accomplish complex tasks. Recently, significant progress has been made in understanding the emergence of modules in both biological and artificial neural systems. How, and under what conditions, the rules of module recombination are represented in these systems remains an open question. Here we present a neural model that can leverage these rules to dramatically speed up learning. We first show that when faced with multiple tasks which share subcomponents, models learn a low-dimensional representation that captures how subcomponents are reused across the task set. These low-dimensional spaces encode the structure that governs how modules should be recombined. Restricting learning to these subspaces greatly reduces the amount of experience needed to acquire a novel task, even when learning from reinforcement on single trials. In some cases, we can leverage the geometric regularities of these representations to reduce learning to a form of hypothesis testing over a small set of discrete points. Finally, we use this theory to model both behavioral and neural data from non-human primates performing a compositional task, and show that key features in this data are consistent with a model in which exploration during learning is restricted to these low-dimensional spaces. Overall, this work shows that the advantages of modularity in neural systems can be greatly improved upon when models represent the structure of module reuse. Both these features working in tandem lead to learning on timescales similar to biological intelligences, and hence provide a model for how such fast, adaptable behavior can emerge from systems of neurons.

The Bayesian Brain hypothesis assumes that cognition relies on internal generative models of the world, yet existing implementations remain constrained by pre-specified, task-specific generative structures and computationally heavy iterative inference schemes. Here, we introduce modular neural state-space models as a scalable realization of the Bayesian Brain, replacing fixed generative structures and pre-specified inference rules with learned world-model components and amortized neural updates. This framework preserves the core commitment to explaining observations through hidden causes while making inference learned and reusable rather than pre-specified and task-specific. Our modular implementation of these models affords learned components to be seamlessly recombined and stacked across superficially different tasks that share similar latent dynamics. Such computational reuse supports zero-shot generalization and predicts selective correlations of inference parameters between tasks. We confirm these key predictions in human behavior, identifying learned and reusable world-model components as a candidate computational principle for flexible cognition.

Animals exhibit a remarkable capacity to adapt to novel challenges on their first attempt. We propose that this zero-shot adaptability requires keeping behavioral outcomes highly predictable and controllable, which is achieved by structuring behavior as short compositions of simple primitives. Continuously across environments, the agent learns to predict the outcomes of primitive sequences in an unsupervised manner. When solving a new task, it optimizes short primitive compositions using differentiable model-predictive control. Constraining behavior to short compositions keeps gradient-based planning tractable, yielding efficiency orders of magnitude beyond traditional methods. Applied to mouse homing-escape behavior, our model explains the emergence of subgoals across arena configurations where conventional models fail. Ablations confirm that the components enabling rapid learning in our model (world models, planning models, primitives, and compositionality) are necessary to reproduce escape trajectories. Our results suggest that optimizing predictable primitive compositions is a core mechanism driving zero-shot adaptability and explaining mouse behavioral phenomena.

The neural representations of prior information about the state of the world are poorly understood¹. Here, to investigate them, we examined brain-wide Neuropixels recordings and widefield calcium imaging collected by the International Brain Laboratory. Mice were trained to indicate the location of a visual grating stimulus, which appeared on the left or right with a prior probability alternating between 0.2 and 0.8 in blocks of variable length. We found that mice estimate this prior probability and thereby improve their decision accuracy. Furthermore, we report that this subjective prior is encoded in at least 20% to 30% of brain regions that, notably, span all levels of processing, from early sensory areas (the lateral geniculate nucleus and primary visual cortex) to motor regions (secondary and primary motor cortex and gigantocellular reticular nucleus) and high-level cortical regions (the dorsal anterior cingulate area and ventrolateral orbitofrontal cortex). This widespread representation of the prior is consistent with a neural model of Bayesian inference involving loops between areas, as opposed to a model in which the prior is incorporated only in decision-making areas. This study offers a brain-wide perspective on prior encoding at cellular resolution, underscoring the importance of using large-scale recordings on a single standardized task.

To thrive in complex environments, animals and artificial agents must learn to act adaptively to maximize fitness and rewards. Such adaptive behaviour can be learned through reinforcement learning¹, a class of algorithms that has been successful at training artificial agents^2,3,4,5 and at characterizing the firing of dopaminergic neurons in the midbrain^6,7,8. In classical reinforcement learning, agents discount future rewards exponentially according to a single timescale, known as the discount factor. Here we explore the presence of multiple timescales in biological reinforcement learning. We first show that reinforcement agents learning at a multitude of timescales possess distinct computational benefits. Next, we report that dopaminergic neurons in mice performing two behavioural tasks encode reward prediction error with a diversity of discount time constants. Our model explains the heterogeneity of temporal discounting in both cue-evoked transient responses and slower timescale fluctuations known as dopamine ramps. Crucially, the measured discount factor of individual neurons is correlated across the two tasks, suggesting that it is a cell-specific property. Together, our results provide a new paradigm for understanding functional heterogeneity in dopaminergic neurons and a mechanistic basis for the empirical observation that humans and animals use non-exponential discounts in many situations^9,10,11,12, and open new avenues for the design of more-efficient reinforcement learning algorithms.

A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.

Efficient musculoskeletal simulators and powerful learning algorithms provide computational tools to tackle the grand challenge of understanding biological motor control. Our winning solution for the inaugural NeurIPS MyoChallenge leverages an approach mirroring human skill learning. Using a novel curriculum learning approach, we trained a recurrent neural network to control a realistic model of the human hand with 39 muscles to rotate two Baoding balls in the palm of the hand. In agreement with data from human subjects, the policy uncovers a small number of kinematic synergies even though it is not explicitly biased towards low-dimensional solutions. However, by selectively inactivating parts of the control signal, we found that more dimensions contribute to the task performance than suggested by traditional synergy analysis. Overall, our work illustrates the emerging possibilities at the interface of musculoskeletal physics engines, reinforcement learning and neuroscience to advance our understanding of biological motor control.

Connecting chemical properties to various wine characteristics is of great interest to the science of olfaction as well as the wine industry. We explored whether Bordeaux wine chemical identities and vintages (harvest year) can be inferred from a common and affordable chemical analysis, namely, a combination of gas chromatography (GC) and electron ionization mass spectrometry. Using 12 vintages (within the 1990–2007 range) from 7 estates of the Bordeaux region, we report that, remarkably, nonlinear dimensionality reduction techniques applied to raw gas chromatograms recover the geography of the Bordeaux region. Using machine learning, we found that we can not only recover the estate perfectly from gas chromatograms, but also the vintage with up to 50% accuracy. Interestingly, we observed that the entire chromatogram is informative with respect to geographic location and age, thus suggesting that the chemical identity of a wine is not defined by just a few molecules but is distributed over a large chemical spectrum. This study demonstrates the remarkable potential of GC analysis to explore fundamental questions about the origin and age of wine.