Selmaan Chettih

Multi-view pose estimation is essential for quantifying animal behavior in scientific research, yet current methods struggle to achieve accurate tracking with limited labeled data and suffer from poor uncertainty estimates. We address these challenges with a flexible framework that can operate with or without camera calibration, combining novel training and post-processing techniques with an uncertainty-aware pseudo-labeling distillation procedure. Our multi-view model processes all camera views jointly using a pretrained vision transformer backbone, and a simulated occlusion technique encourages the model to learn robust cross-view correspondences without requiring camera parameters. When camera parameters are available, 3D data augmentations and a triangulation-based loss further encourage geometric consistency. We extend the Ensemble Kalman Smoother (EKS) post-processor to the nonlinear case, leveraging camera geometry, and introduce a variance inflation technique that detects cross-view inconsistencies and corrects overconfident predictions. We validate our approach on five datasets spanning three species (fly, mouse, bird), including a multi-animal dataset with two visually distinct individuals; the proposed pipeline consistently outperforms existing methods across datasets. We demonstrate how these improvements translate to downstream scientific analyses using data from the International Brain Laboratory, showing improved unsupervised behavioral clustering and neural decoding of paw kinematics with just 200 labeled frames. To facilitate adoption, we developed a browser-based, cloud-compatible user interface that supports the full life cycle of multi-view pose estimation, from labeling and model training to post-processing with EKS and diagnostic visualizations.

Forming an episodic memory requires binding together disparate elements that co-occur in a single experience. One model of this process is that neurons representing different components of a memory bind to an ‘index’ — a subset of neurons unique to that memory. Evidence for this model has recently been found in chickadees, which use hippocampal memory to store and recall locations of cached food. Chickadee hippocampus produces sparse, high-dimensional patterns (‘barcodes’) that uniquely specify each caching event. Unexpectedly, the same neurons that participate in barcodes also exhibit conventional place tuning. It is unknown how barcode activity is generated, and what role it plays in memory formation and retrieval. It is also unclear how a memory index (e.g. barcodes) could function in the same neural population that represents memory content (e.g. place). Here, we design a biologically plausible model that generates barcodes and uses them to bind experiential content. Our model generates barcodes from place inputs through the chaotic dynamics of a recurrent neural network and uses Hebbian plasticity to store barcodes as attractor states. The model matches experimental observations that memory indices (barcodes) and content signals (place tuning) are randomly intermixed in the activity of single neurons. We demonstrate that barcodes reduce memory interference between correlated experiences. We also show that place tuning plays a complementary role to barcodes, enabling flexible, contextually appropriate memory retrieval. Finally, our model is compatible with previous models of the hippocampus as generating a predictive map. Distinct predictive and indexing functions of the network are achieved via an adjustment of global recurrent gain. Our results suggest how the hippocampus may use barcodes to resolve fundamental tensions between memory specificity (pattern separation) and flexible recall (pattern completion) in general memory systems.

Forming an episodic memory requires binding together disparate elements that co-occur in a single experience. One model of this process is that neurons representing different components of a memory bind to an “index” — a subset of neurons unique to that memory. Evidence for this model has recently been found in chickadees, which use hippocampal memory to store and recall locations of cached food. Chickadee hippocampus produces sparse, high-dimensional patterns (“barcodes”) that uniquely specify each caching event. Unexpectedly, the same neurons that participate in barcodes also exhibit conventional place tuning. It is unknown how barcode activity is generated, and what role it plays in memory formation and retrieval. It is also unclear how a memory index (e.g. barcodes) could function in the same neural population that represents memory content (e.g. place). Here, we design a biologically plausible model that generates barcodes and uses them to bind experiential content. Our model generates barcodes from place inputs through the chaotic dynamics of a recurrent neural network and uses Hebbian plasticity to store barcodes as attractor states. The model matches experimental observations that memory indices (barcodes) and content signals (place tuning) are randomly intermixed in the activity of single neurons. We demonstrate that barcodes reduce memory interference between correlated experiences. We also show that place tuning plays a complementary role to barcodes, enabling flexible, contextually-appropriate memory retrieval. Finally, our model is compatible with previous models of the hippocampus as generating a predictive map. Distinct predictive and indexing functions of the network are achieved via an adjustment of global recurrent gain. Our results suggest how the hippocampus may use barcodes to resolve fundamental tensions between memory specificity (pattern separation) and flexible recall (pattern completion) in general memory systems.

Forming an episodic memory requires binding together disparate elements that co-occur in a single experience. One model of this process is that neurons representing different components of a memory bind to an “index” — a subset of neurons unique to that memory. Evidence for this model has recently been found in chickadees, which use hippocampal memory to store and recall locations of cached food. Chickadee hippocampus produces sparse, high-dimensional patterns (“barcodes”) that uniquely specify each caching event. Unexpectedly, the same neurons that participate in barcodes also exhibit conventional place tuning. It is unknown how barcode activity is generated, and what role it plays in memory formation and retrieval. It is also unclear how a memory index (e.g. barcodes) could function in the same neural population that represents memory content (e.g. place). Here, we design a biologically plausible model that generates barcodes and uses them to bind experiential content. Our model generates barcodes from place inputs through the chaotic dynamics of a recurrent neural network and uses Hebbian plasticity to store barcodes as attractor states. The model matches experimental observations that memory indices (barcodes) and content signals (place tuning) are randomly intermixed in the activity of single neurons. We demonstrate that barcodes reduce memory interference between correlated experiences. We also show that place tuning plays a complementary role to barcodes, enabling flexible, contextually-appropriate memory retrieval. Finally, our model is compatible with previous models of the hippocampus as generating a predictive map. Distinct predictive and indexing functions of the network are achieved via an adjustment of global recurrent gain. Our results suggest how the hippocampus may use barcodes to resolve fundamental tensions between memory specificity (pattern separation) and flexible recall (pattern completion) in general memory systems.

Episodic memory, or memory of experienced events, is a critical function of the hippocampus ^1–3 . It is therefore important to understand how hippocampal activity represents specific events in an animal’s life. We addressed this question in chickadees – specialist food-caching birds that hide food at scattered locations and use memory to find their caches later in time ^{4, 5} . We performed high-density neural recordings in the hippocampus of chickadees as they cached and retrieved seeds in a laboratory arena. We found that each caching event was represented by a burst of firing in a unique set of hippocampal neurons. These ‘barcode-like’ patterns of activity were sparse (6, 7 , as well as location-independent responses to cached seeds. We propose that barcodes are signatures of episodic memories evoked during memory recall. These patterns assign a unique identifier to each event and may be a mechanism for rapid formation and storage of many non-interfering memories.

Investigating how visual inputs are encoded in visual cortex is important for elucidating the roles of cell populations in circuit computations. We here use a recently developed, large-scale model of mouse primary visual cortex (V1) and perturb both single neurons as well as functional- and cell-type defined population of neurons to mimic equivalent optogenetic perturbations. First, perturbations were performed to study the functional roles of layer 2/3 excitatory neurons in inter-laminar interactions. We observed activity changes consistent with the canonical cortical model (Douglas and Martin 1991). Second, single neuron perturbations in layer 2/3 revealed a center-surround inhibition-dominated effect, consistent with recent experiments. Finally, perturbations of multiple excitatory layer 2/3 neurons during visual stimuli of varying contrasts indicated that the V1 model has both efficient and robust coding features. The circuit transitions from predominantly broad like-to-like inhibition at high contrasts to predominantly specific like-to-like excitation at low contrasts. These in silico results demonstrate how the circuit can shift from redundancy reduction to robust codes as a function of stimulus contrast.

Calcium imaging is a key method in neuroscience for investigating patterns of neuronal activityin vivo. Still, existing algorithms to detect and extract activity signals from calcium-imaging movies have major shortcomings. We introduce the HNCcorr algorithm for cell identification in calcium-imaging datasets that addresses these shortcomings. HNCcorr relies on the combinatorial clustering problem HNC (Hochbaum’s Normalized Cut), which is similar to the Normalized Cut problem of Shi and Malik, a well known problem in image segmentation. HNC identifies cells as coherent clusters of pixels that are highly distinct from the remaining pixels. HNCcorr guarantees a globally optimal solution to the underlying optimization problem as well as minimal dependence on initialization techniques. HNCcorr also uses a new method, called “similarity squared”, for measuring similarity between pixels in calcium-imaging movies. The effectiveness of HNCcorr is demonstrated by its top performance on the Neurofinder cell identification benchmark. We believe HNCcorr is an important addition to the toolbox for analysis of calcium-imaging movies.

Eyeblink conditioning in restrained rabbits has served as an excellent model of cerebellar-dependent motor learning for many decades. In mice, the role of the cerebellum in eyeblink conditioning is less clear and remains controversial, partly because learning appears to engage fear-related circuits and lesions of the cerebellum do not abolish the learned behavior completely. Furthermore, experiments in mice are performed using freely moving systems, which lack the stability necessary for mapping out the essential neural circuitry with electrophysiological approaches. We have developed a novel apparatus for eyeblink conditioning in head-fixed mice. Here, we show that the performance of mice in our apparatus is excellent and that the learned behavior displays two hallmark features of cerebellar-dependent eyeblink conditioning in rabbits: (1) gradual acquisition; and (2) adaptive timing of conditioned movements. Furthermore, we use a combination of pharmacological inactivation, electrical stimulation, single-unit recordings, and targeted microlesions to demonstrate that the learned behavior is completely dependent on the cerebellum and to pinpoint the exact location in the deep cerebellar nuclei that is necessary. Our results pave the way for using eyeblink conditioning in head-fixed mice as a platform for applying next-generation genetic tools to address molecular and circuit-level questions about cerebellar function in health and disease.