Ilana B. Witten

Multi-view pose estimation is essential for quantifying animal behavior in scientific research, yet current methods struggle to achieve accurate tracking with limited labeled data and suffer from poor uncertainty estimates. We address these challenges with a flexible framework that can operate with or without camera calibration, combining novel training and post-processing techniques with an uncertainty-aware pseudo-labeling distillation procedure. Our multi-view model processes all camera views jointly using a pretrained vision transformer backbone, and a simulated occlusion technique encourages the model to learn robust cross-view correspondences without requiring camera parameters. When camera parameters are available, 3D data augmentations and a triangulation-based loss further encourage geometric consistency. We extend the Ensemble Kalman Smoother (EKS) post-processor to the nonlinear case, leveraging camera geometry, and introduce a variance inflation technique that detects cross-view inconsistencies and corrects overconfident predictions. We validate our approach on five datasets spanning three species (fly, mouse, bird), including a multi-animal dataset with two visually distinct individuals; the proposed pipeline consistently outperforms existing methods across datasets. We demonstrate how these improvements translate to downstream scientific analyses using data from the International Brain Laboratory, showing improved unsupervised behavioral clustering and neural decoding of paw kinematics with just 200 labeled frames. To facilitate adoption, we developed a browser-based, cloud-compatible user interface that supports the full life cycle of multi-view pose estimation, from labeling and model training to post-processing with EKS and diagnostic visualizations.

Decision making is traditionally thought to be mediated by neurons that accumulate evidence through persistent activity. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially, rather than persistently, with the subset of neurons in the sequence depending on the animal’s choice. We developed two candidate circuit models in which neurons are active sequentially and transfer evidence faithfully to the next active population. One model encodes evidence in the relative firing of two competing chains of neurons, and the other in the network location of a stereotyped, bump-like pattern of neural activity. Neural recordings from four brain regions during an evidence accumulation task revealed that different regions displayed evidence tuning consistent with different candidate models. This work provides a mechanistic explanation for how graded information may be precisely accumulated within and transferred between neural populations, and suggests that different brain regions may accumulate evidence through different circuit mechanisms.

Summary: Individual responses to chronic stress vary, with some individuals remaining resilient while others exhibit susceptibility. The ventral tegmental area (VTA), a region involved in reward learning, and the lateral habenula (LHb), a region involved in aversive learning, have been implicated in the pathophysiology of stress-related mood disorders. Here, we seek to understand the molecular adaptations in these regions at the level of single cells that mediate susceptibility versus resilience. In particular, it remains unclear whether, at the level of gene expression, different cell types within different brain regions mediate stress susceptibility versus resilience, or if these phenotypes are mediated by distinct trajectories within the same cell types. To address this gap, we performed single-nucleus RNA-sequencing of LHb and VTA of mice subjected to chronic social defeat stress. While we found minimal gene expression changes in the LHb after stress, the VTA exhibited widespread, cell type-specific transcriptional remodeling in resilient individuals and few gene expression changes in susceptible individuals. Across VTA cell types, resilience was associated with the coordinated upregulation of genes involved in intercellular signaling and neural communication, with maintenance of receptor-ligand interaction strength in resilience that was not present in susceptibility. Within VTA neurons, gene expression changes were most prominent in glutamatergic and dopaminergic clusters. Multivariate analyses of dopamine and glutamate subclusters showed that resilient neurons diverged more from control than susceptible neurons, but along a similar trajectory, supporting a model in which resilience reflects greater stress-related adaptations in these cell types. Together, these findings highlight the VTA as a key site of molecular plasticity in stress resilience and therefore a potential therapeutic target.

How does the activity of midbrain dopamine (DA) neurons reinforce actions? A prominent hypothesis is that the activity of ventral tegmental area (VTA) DA neurons instructs representations of predicted reward, or value, in downstream neurons1. To directly test this model, we performed comprehensive striatal recordings in mice engaged in a trial-and-error probabilistic learning task where they continuously adapted their choices to obtain a reward of optogenetic stimulation of VTA DA neurons (paired with an auditory cue). We then assessed neural representations of action values (estimated from a behavioral model), revealing for the first time that VTA DA stimulation is sufficient to generate downstream neural correlates of action value. Surprisingly, these value correlates were strongest in the intermediate caudoputamen (CP) and weakest in the nucleus accumbens (NAc), despite NAc being the major projection target of VTA DA neurons2,3. This was true not only for the value of each choice, but also for state value (reward expectation) and relative value (the decision variable). However, value representations were differentially organized within the intermediate CP, with ventromedial domains (which receive inputs from orbitofrontal cortex) preferentially encoding state value and dorsolateral domains (which receive inputs from motor cortex) preferentially encoding relative value. A difference in learning rate for the value computation between NAc and CP did not explain the relatively weak value correlates in NAc. Instead, we found that VTA DA stimulation was sufficient to produce learned neural responses to the stimulation-paired auditory cue throughout the striatum, including in the NAc, and that animals work for this cue rather than for VTA DA stimulation itself. Overall, this suggests that VTA DA neurons support trial-and-error learning indirectly, by making stimuli valuable ('conditioned reinforcers'), which in turn support the generation of action value representations in the CP.

The neural representations of prior information about the state of the world are poorly understood¹. Here, to investigate them, we examined brain-wide Neuropixels recordings and widefield calcium imaging collected by the International Brain Laboratory. Mice were trained to indicate the location of a visual grating stimulus, which appeared on the left or right with a prior probability alternating between 0.2 and 0.8 in blocks of variable length. We found that mice estimate this prior probability and thereby improve their decision accuracy. Furthermore, we report that this subjective prior is encoded in at least 20% to 30% of brain regions that, notably, span all levels of processing, from early sensory areas (the lateral geniculate nucleus and primary visual cortex) to motor regions (secondary and primary motor cortex and gigantocellular reticular nucleus) and high-level cortical regions (the dorsal anterior cingulate area and ventrolateral orbitofrontal cortex). This widespread representation of the prior is consistent with a neural model of Bayesian inference involving loops between areas, as opposed to a model in which the prior is incorporated only in decision-making areas. This study offers a brain-wide perspective on prior encoding at cellular resolution, underscoring the importance of using large-scale recordings on a single standardized task.

Understanding brain function relies on the collective work of many labs generating reproducible results. However, reproducibility has not been systematically assessed within the context of electrophysiological recordings during cognitive behaviors. To address this, we formed a multi-lab collaboration using a shared, open-source behavioral task and experimental apparatus. Experimenters in 10 laboratories repeatedly targeted Neuropixels probes to the same location (spanning secondary visual areas, hippocampus, and thalamus) in mice making decisions; this generated a total of 121 experimental replicates, a unique dataset for evaluating reproducibility of electrophysiology experiments. Despite standardizing both behavioral and electrophysiological procedures, some experimental outcomes were highly variable. A closer analysis uncovered that variability in electrode targeting hindered reproducibility, as did the limited statistical power of some routinely used electrophysiological analyses, such as single-neuron tests of modulation by individual task parameters. Reproducibility was enhanced by histological and electrophysiological quality-control criteria. Our observations suggest that data from systems neuroscience is vulnerable to a lack of reproducibility, but that across-lab standardization, including metrics we propose, can serve to mitigate this.

Animals learn the value of foods on the basis of their postingestive effects and thereby develop aversions to foods that are toxic^1–10 and preferences to those that are nutritious^11–13. However, it remains unclear how the brain is able to assign credit to flavours experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here we reveal an unexpected role for the postingestive reactivation of neural flavour representations in this temporal credit-assignment process. To begin, we leverage the fact that mice learn to associate novel^14,15, but not familiar, flavours with delayed gastrointestinal malaise signals to investigate how the brain represents flavours that support aversive postingestive learning. Analyses of brain-wide activation patterns reveal that a network of amygdala regions is unique in being preferentially activated by novel flavours across every stage of learning (consumption, delayed malaise and memory retrieval). By combining high-density recordings in the amygdala with optogenetic stimulation of malaise-coding hindbrain neurons, we show that delayed malaise signals selectively reactivate flavour representations in the amygdala from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts the strengthening of flavour responses upon memory retrieval, which in turn leads to stabilization of the population-level representation of the recently consumed flavour. By contrast, flavour representations in the amygdala degrade in the absence of unexpected postingestive consequences. Thus, we demonstrate that postingestive reactivation and plasticity of neural flavour representations may support learning from delayed feedback.

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary ‘feature-specific RPE’ model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal’s moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.