Walter Senn

Learning to detect, identify, or select stimuli is an essential requirement of many behavioral tasks. In real-life situations, relevant and nonrelevant stimuli are often embedded in a continuous sensory stream, presumably represented by different segments of neural activity. Here we introduce a spiking network model that can discover action-relevant stimuli in an unsegmented sensory stream of spike trains. The model uses a biologically plausible plasticity rule and learns from the reinforcement of correct decisions taken at the right time. Learning is fully online and is faster for larger population size; it allows for a wide spectrum of neural-encoding strategies and can segment cortical spike patterns recorded from behaving animals. Based on these results, the proposed model provides a biologically plausible framework for reinforcement learning in the absence of prior information on the identity, relevance, and timing of input stimuli embedded in a continuous spatiotemporal stream.

Understanding the variability of the environment is essential to function in everyday life. The brain must hence take uncertainty into account when updating its internal model of the world. The basis for updating the model are prediction errors that arise from a difference between the current model and new sensory experiences. Although prediction error neurons have been identified in layer 2/3 of diverse brain areas, how uncertainty modulates these errors and hence learning is, however, unclear. Here, we use a normative approach to derive how uncertainty should modulate prediction errors and postulate that layer 2/3 neurons represent uncertainty-modulated prediction errors (UPE). We further hypothesise that the layer 2/3 circuit calculates the UPE through the subtractive and divisive inhibition by different inhibitory cell types. By implementing the calculation of UPEs in a microcircuit model, we show that different cell types can compute the means and variances of the stimulus distribution. With local activity-dependent plasticity rules, these computations can be learned context-dependently, and allow the prediction of upcoming stimuli and their distribution. Finally, the mechanism enables an organism to optimise its learning strategy via adaptive learning rates.

One of the most fundamental laws of physics is the principle of least action. Motivated by its predictive power, we introduce a neuronal least-action principle for cortical processing of sensory streams to produce appropriate behavioral outputs in real time. The principle postulates that the voltage dynamics of cortical pyramidal neurons prospectively minimizes the local somato-dendritic mismatch error within individual neurons. For output neurons, the principle implies minimizing an instantaneous behavioral error. For deep network neurons, it implies the prospective firing to overcome integration delays and correct for possible output errors right in time. The neuron-specific errors are extracted in the apical dendrites of pyramidal neurons through a cortical microcircuit that tries to explain away the feedback from the periphery, and correct the trajectory on the fly. Any motor output is in a moving equilibrium with the sensory input and the motor feedback during the ongoing sensory-motor transform. Online synaptic plasticity reduces the somatodendritic mismatch error within each cortical neuron and performs gradient descent on the output cost at any moment in time. The neuronal least-action principle offers an axiomatic framework to derive local neuronal and synaptic laws for global real-time computation and learning in the brain.

How sensory information is interpreted depends on context. Yet, how context shapes sensory processing in the brain, remains elusive. To investigate this question we combined computational modeling and in vivo functional imaging of cortical neurons in mice during reversal learning of a tactile sensory discrimination task. During learning, layer 2/3 somatosensory neurons enhanced their response to reward-predictive stimuli, explainable as gain amplification from apical dendrites. Reward-prediction errors were reduced and confidence in the outcome prediction increased. Upon rule-reversal, the lateral orbitofrontal cortex, through disinhibitory VIP interneurons, encoded a context-prediction error signaling a loss of confidence. The hierarchy of prediction errors in cortical areas is mirrored in top-down signals modulating apical activity in the primary sensory cortex. Our model explains how contextual changes are detected in the brain and how errors in different cortical regions interact to reshape and update the sensory representation.

The importance of sleep for healthy brain function is widely acknowledged. However, it remains unclear how the internal generation of dreams might facilitate cognitive processes. In this perspective, we review a computational approach inspired by artificial intelligence that proposes a framework for how dreams occurring during rapid-eye-movement (REM) sleep can contribute to learning and creativity. In this framework, REM dreams are characterized by an adversarial process that, against the dream reality, tells a discriminator network to classify the internally created sensory activity as real. Such an adversarial dreaming process is shown to facilitate the emergence of real-world semantic representations in higher cortical areas. We further discuss the potential contributions of adversarial dreaming beyond learning, such as balancing fantastic and realistic dream elements and facilitating the occurrence of creative insights. We characterize non-REM (NREM) dreams, where a single hippocampal memory is replayed at a time, as serving the complementary role of improving the robustness of cortical representations to environmental perturbations. We finally explain how subjects can become aware of the adversarial REM dreams, but less of the NREM dreams, and how content- and state-awareness in wake, dream, and lucid dreaming may appear.

The importance of sleep for healthy brain function is widely acknowledged. However, it remains unclear how the internal generation of dreams might facilitate cognitive processes. In this perspective we review a computational approach inspired by artificial intelligence that proposes a framework for how dreams occurring during rapid-eye-movement (REM) sleep can contribute to learning and creativity. In this framework, REM dreams are characterized by an adversarial process that, against the dream reality, tell a discriminator network to classify the internally created sensory activity as real. Such an adversarial dreaming process is shown to facilitate the emergence of real-world semantic representations in higher cortical areas. We further discuss the potential contributions of adversarial dreaming beyond learning, such as balancing fantastic and realistic dream elements, and facilitating the occurrence of creative insights. We characterize non-REM (NREM) dreams, where a single hippocampal memory is replayed at a time, as serving a complementary role of improving the robustness of cortical representations to environmental perturbations. We finally explain how subjects can become aware of the adversarial REM dreams, but less of the NREM dreams, and how the awareness phenomenon of wake, dream and lucid dreaming may appear.

One of the most fundamental laws of physics is the principle of least action. Motivated by its predictive power, we introduce a neuronal least-action principle for cortical processing of sensory streams to produce appropriate behavioural outputs in real time. The principle postulates that the voltage dynamics of cortical pyramidal neurons prospectively minimize the local somato-dendritic mismatch error within individual neurons. For motor output neurons, it implies minimizing an instantaneous behavioural error. For deep network neurons, it implies a prospective firing to overcome integration delays and correct for possible output errors right in time. The neuron-specific errors are extracted in the apical dendrites of pyramidal neurons through a cortical microcircuit that tries to explain away the feedback from the periphery, and correct the trajectory on the fly. Any motor output is in a moving equilibrium with the sensory inputs and the motor feedback during the whole sensory-motor trajectory. Ongoing synaptic plasticity reduces the somato-dendritic mismatch error within each cortical neuron and performs gradient descent on the output cost at any moment in time. The neuronal least-action principle offers an axiomatic framework to derive local neuronal and synaptic dynamics for global real-time computation and learning in the brain and in physical substrates in general.

Uncertainty is omnipresent. While humans and other animals take uncertainty into account during decision making, it remains unclear how it is represented in cortex. To investigate the effect of stimulus reliability on uncertainty representation in cortical neurons, we analyzed single unit activity data recorded in mouse PPC, while animals performed a multisensory change detection task. We further used simulation-based inference (SBI) to infer membrane potential statistics underlying the spiking activity. Our analysis shows that stimulus changes increase spiking rate while decreasing its variability. The inferred membrane potential statistics suggest that PPC neurons decrease their membrane potential variability in response to task relevant stimuli. Furthermore, more perceptually reliable stimuli lead to a larger decrease in membrane potential variability than less reliable ones. These findings suggest that individual cortical neurons track uncertainty, providing Bayesian benefits for downstream computations.

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contextual state. Here, we demonstrate that dendritic N-Methyl-D-Aspartate spikes can, within physiological constraints, implement contextual modulation of feedforward processing. Such neuron-specific modulations exploit prior knowledge, encoded in stable feedforward weights, to achieve transfer learning across contexts. In a network of biophysically realistic neuron models with context-independent feedforward weights, we show that modulatory inputs to dendritic branches can solve linearly nonseparable learning problems with a Hebbian, error-modulated learning rule. We also demonstrate that local prediction of whether representations originate either from different inputs, or from different contextual modulations of the same input, results in representation learning of hierarchical feedforward weights across processing layers that accommodate a multitude of contexts.