Talmo D. Pereira
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Talmo D. Pereira has a total of 320 co-authors. Here are the top co-authors by number of shared publications:
- Mala Murthy
Princeton Neuroscience Institute, Princeton University
11 shared publications - Joshua W. Shaevitz
Joseph Henry Laboratories of Physics and Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ 08544, USA
7 shared publications - Kay M. Tye
Salk Institute for Biological Studies, Howard Hughes Medical Institute
7 shared publications - Hao Li
Salk Institute for Biological Studies, La Jolla, CA, USA
6 shared publications - Laurel R. Keyes
Salk Institute for Biological Studies
6 shared publications - Romy Wichmann
Salk Institute for Biological Studies, La Jolla, USA
6 shared publications
Latest Publications
- Open Access
LUC3D: Multi-animal 3D pose tracking in the browser
Animal behavior unfolds in three dimensions over time, yet accessible scientific tools for analyzing it in its natural form remain limited. Uncertainty from popular 2D pose methods can be refined with multi-camera approaches to produce more accurate 3D estimates. This paper presents LUC3D (Label Unification and Correspondence in 3D Annotation GUI), a browser-based 3D GUI for reducing the amount of manual labor required for labeling, proofreading, and multi-camera multi-animal tracking while requiring no installation to maximize accessibility. As proof-of-concept we present two datasets of proofread multi-camera 3D triangulated pose data of mouse social behavior, a protocol for generating 3D consistent labels, and a cross-view re-identification tracking algorithm for multi-animal multi-camera pose data. This tool allows for the widespread adoption of 3D pose estimation techniques for studying social behavior while reducing the cost of entry for neuroscience labs across the world.
Open AccessLUC3D: Multi-animal 3D pose tracking in the browser
Animal behavior unfolds in three dimensions over time, yet accessible scientific tools for analyzing it in its natural form remain limited. Uncertainty from popular 2D pose methods can be refined with multi-camera approaches to produce more accurate 3D estimates. This paper presents LUC3D (Label Unification and Correspondence in 3D Annotation GUI), a browser-based 3D GUI for reducing the amount of manual labor required for labeling, proofreading, and multi-camera multi-animal tracking while requiring no installation to maximize accessibility. As proof-of-concept we present two datasets of proofread multi-camera 3D triangulated pose data of mouse social behavior, a protocol for generating 3D consistent labels, and a cross-view re-identification tracking algorithm for multi-animal multi-camera pose data. This tool allows for the widespread adoption of 3D pose estimation techniques for studying social behavior while reducing the cost of entry for neuroscience labs across the world.
- Open Access
- 25 August 2026
- Published by Springer Nature in Nature Neuroscience
- Vol. 29 (10) , 2513-2522
Social isolation recruits amygdala–medial prefrontal cortex projections to escalate alcohol drinking in male mice
Social isolation profoundly alters motivation and increases vulnerability to alcohol misuse in humans, yet the underlying neural mechanisms remain unclear. Here we show that isolation escalates alcohol drinking in male mice but suppresses it in females. Whole-cell recordings revealed that neurons in the basolateral amygdala projecting to the medial prefrontal cortex (BLA–mPFC) track alcohol intake in both sexes. Isolation increased BLA–mPFC excitability in males but decreased it in females, mirroring their opposite behavioral adaptations. Given this divergence, we focused subsequent mechanistic studies on males to isolate neural pathway-level drivers of escalated alcohol intake. Cellular-resolution calcium imaging showed that activity in BLA–mPFC neurons encodes and predicts alcohol drinking, and optogenetic activation of this pathway increased alcohol intake. Simultaneous optogenetics and calcium imaging revealed that BLA–mPFC stimulation enhanced mPFC neuronal responses to alcohol, mimicking isolation-induced activity patterns, while photoinhibition reduced drinking in isolated mice. Together, these findings identify a BLA–mPFC pathway mechanism through which social isolation reconfigures prefrontal processing to promote alcohol intake.
- 25 August 2026
- Published by Springer Nature in Nature Neuroscience
- Vol. 29 (10) , 2513-2522
Open AccessSocial isolation recruits amygdala–medial prefrontal cortex projections to escalate alcohol drinking in male mice
Social isolation profoundly alters motivation and increases vulnerability to alcohol misuse in humans, yet the underlying neural mechanisms remain unclear. Here we show that isolation escalates alcohol drinking in male mice but suppresses it in females. Whole-cell recordings revealed that neurons in the basolateral amygdala projecting to the medial prefrontal cortex (BLA–mPFC) track alcohol intake in both sexes. Isolation increased BLA–mPFC excitability in males but decreased it in females, mirroring their opposite behavioral adaptations. Given this divergence, we focused subsequent mechanistic studies on males to isolate neural pathway-level drivers of escalated alcohol intake. Cellular-resolution calcium imaging showed that activity in BLA–mPFC neurons encodes and predicts alcohol drinking, and optogenetic activation of this pathway increased alcohol intake. Simultaneous optogenetics and calcium imaging revealed that BLA–mPFC stimulation enhanced mPFC neuronal responses to alcohol, mimicking isolation-induced activity patterns, while photoinhibition reduced drinking in isolated mice. Together, these findings identify a BLA–mPFC pathway mechanism through which social isolation reconfigures prefrontal processing to promote alcohol intake.
- Open Access
- 19 August 2026
- Published by Springer Nature in Nature
- Vol. 658 (8134) , 174-185
Asymmetric prefrontal representations for leader–follower dynamics
- 19 August 2026
- Published by Springer Nature in Nature
- Vol. 658 (8134) , 174-185
Open AccessAsymmetric prefrontal representations for leader–follower dynamics
- Open Access
Spatial imprints of emergent cardiomyocyte states in the pressure-overloaded heart
Summary: Resilience to cardiac stress is essential for health, yet the relationship between cardiomyocyte (CM) stress response and local microenvironment remains unclear. Here, we combined MERFISH spatial transcriptome profiling with Cellouette , an improved cell segmentation method, to determine CM-microenvironment relationships in a mouse model of ventricular pressure overload. We report the shape, transcription profile, spatial organization, and physical connectivity for >400,000 cells across stressed and healthy tissues. Under stress, CMs adopted a spectrum of emergent transcriptional states, with advanced states marked by a metabolic and pro-fibrotic shift. To discover CM-environment relationships, we performed a network analysis of physical cell connectivity combined with cell-type-specific profiling. We found that pro-fibrotic CM progression was tightly linked to distinct local microenvironments, and CM metabolic shifts could be inferred from transcriptional patterns in neighboring non-CM cells, revealing microenvironmental imprints of disease. We thus provide a resource for understanding the heterogeneity of outcome during cardiac pressure overload. Highlights: Cellouette provides accurate segmentation for single-cell spatial transcriptomics in cardiac tissue. Pressure overload creates spatial gradients of cardiomyocyte pro-fibrotic states. Cardiomyocyte pro-fibrotic progression is linked to changes in local cell composition and gene expression. Transcriptional states of non-muscle cells predict metabolic state of adjacent cardiomyocytes.
Open AccessSpatial imprints of emergent cardiomyocyte states in the pressure-overloaded heart
Summary: Resilience to cardiac stress is essential for health, yet the relationship between cardiomyocyte (CM) stress response and local microenvironment remains unclear. Here, we combined MERFISH spatial transcriptome profiling with Cellouette , an improved cell segmentation method, to determine CM-microenvironment relationships in a mouse model of ventricular pressure overload. We report the shape, transcription profile, spatial organization, and physical connectivity for >400,000 cells across stressed and healthy tissues. Under stress, CMs adopted a spectrum of emergent transcriptional states, with advanced states marked by a metabolic and pro-fibrotic shift. To discover CM-environment relationships, we performed a network analysis of physical cell connectivity combined with cell-type-specific profiling. We found that pro-fibrotic CM progression was tightly linked to distinct local microenvironments, and CM metabolic shifts could be inferred from transcriptional patterns in neighboring non-CM cells, revealing microenvironmental imprints of disease. We thus provide a resource for understanding the heterogeneity of outcome during cardiac pressure overload. Highlights: Cellouette provides accurate segmentation for single-cell spatial transcriptomics in cardiac tissue. Pressure overload creates spatial gradients of cardiomyocyte pro-fibrotic states. Cardiomyocyte pro-fibrotic progression is linked to changes in local cell composition and gene expression. Transcriptional states of non-muscle cells predict metabolic state of adjacent cardiomyocytes.
- Open Access
Automated behavioral segmentation and markerless pose tracking of mice during spaceflight
The NASA Rodent Habitat aboard the International Space Station enabled long-duration studies of behavioral responses to spaceflight, but video-based behavioral analysis has relied on laborious manual annotation. No study has tested whether deep learning tools can automate this analysis under the demanding imaging conditions of orbital vivaria. We applied pose estimation (SLEAP) and behavioral segmentation (DeepEthogram) to archival footage from the Rodent Research-1 mission. Nine labelers annotated 3,249 pose labels across 2,063 frames, and three behaviorists labeled 411,194 frames across 66 videos. Pose tracking accuracy approximated human inter-annotator variability despite progressive lens soiling, grid occlusions, and spherical aberration. Behavioral classification across eight categories achieved accuracy of 0.86-0.90 and suggests progressive behavioral adaptations to microgravity. Kinematic reconstruction of circling estimated centripetal accelerations periodically approaching 1 g . This is the first application of deep learning-based pose estimation and behavioral segmentation to rodents in spaceflight, establishing benchmarks for future monitoring systems. One-Sentence Summary: Deep learning-based pose estimation and behavioral segmentation of ISS video reveal progressive behavioral adaptations of mice to microgravity
Open AccessAutomated behavioral segmentation and markerless pose tracking of mice during spaceflight
The NASA Rodent Habitat aboard the International Space Station enabled long-duration studies of behavioral responses to spaceflight, but video-based behavioral analysis has relied on laborious manual annotation. No study has tested whether deep learning tools can automate this analysis under the demanding imaging conditions of orbital vivaria. We applied pose estimation (SLEAP) and behavioral segmentation (DeepEthogram) to archival footage from the Rodent Research-1 mission. Nine labelers annotated 3,249 pose labels across 2,063 frames, and three behaviorists labeled 411,194 frames across 66 videos. Pose tracking accuracy approximated human inter-annotator variability despite progressive lens soiling, grid occlusions, and spherical aberration. Behavioral classification across eight categories achieved accuracy of 0.86-0.90 and suggests progressive behavioral adaptations to microgravity. Kinematic reconstruction of circling estimated centripetal accelerations periodically approaching 1 g . This is the first application of deep learning-based pose estimation and behavioral segmentation to rodents in spaceflight, establishing benchmarks for future monitoring systems. One-Sentence Summary: Deep learning-based pose estimation and behavioral segmentation of ISS video reveal progressive behavioral adaptations of mice to microgravity
- Open Access
- 4 December 2025
- Published by Springer Nature in Nature Methods
- Vol. 23 (1) , 236-247
Deep Imputation for Skeleton data (DISK) for behavioral science
Pose estimation methods and motion capture systems have opened doors to quantitative measurements of animal kinematics. While animal behavior experiments are expensive and complex, tracking errors sometimes make large portions of the experimental data unusable. Here our deep learning method, Deep Imputation for Skeleton data (DISK), uncovers dependencies between keypoints and their dynamics to impute missing tracking data without the help of any manual annotations. We demonstrate the utility and performance of DISK on seven animal skeletons including multi-animal setups. The imputed recordings allow us to detect more episodes of motion, such as steps, and obtain more statistically robust results when comparing these episodes between experimental conditions. In addition, by learning to impute the missing content, DISK learns meaningful representations of the data capturing, for example, underlying actions. This stand-alone imputation package, available at https://github.com/bozeklab/DISK.git/ , is applicable to outputs of tracking methods (marker-based or markerless) and allows for varied types of downstream analysis.
- 4 December 2025
- Published by Springer Nature in Nature Methods
- Vol. 23 (1) , 236-247
Open AccessDeep Imputation for Skeleton data (DISK) for behavioral science
Pose estimation methods and motion capture systems have opened doors to quantitative measurements of animal kinematics. While animal behavior experiments are expensive and complex, tracking errors sometimes make large portions of the experimental data unusable. Here our deep learning method, Deep Imputation for Skeleton data (DISK), uncovers dependencies between keypoints and their dynamics to impute missing tracking data without the help of any manual annotations. We demonstrate the utility and performance of DISK on seven animal skeletons including multi-animal setups. The imputed recordings allow us to detect more episodes of motion, such as steps, and obtain more statistically robust results when comparing these episodes between experimental conditions. In addition, by learning to impute the missing content, DISK learns meaningful representations of the data capturing, for example, underlying actions. This stand-alone imputation package, available at https://github.com/bozeklab/DISK.git/ , is applicable to outputs of tracking methods (marker-based or markerless) and allows for varied types of downstream analysis.
- Open Access
- 3 June 2025
- Published by Springer Nature in Research Square
Social Exclusion Amplifies Behavioral Responses to Physical Pain via Insular Neuromodulation
The “Pain Overlap Theory”1 proposes that the experience of social pain overlaps with and amplifies the experience of physical pain by sharing parts of the same underlying processing systems 2–6. In humans, the insular cortex has been implicated in this overlap of physical and social pain, but a mechanistic link has not been made2,4,5,7–9. To determine whether social pain can subsequently impact responses to nociceptive stimuli via convergent electrical signals (spikes) or convergent chemical signals (neuromodulators), we designed a novel Social Exclusion paradigm termed the Fear of Missing Out (FOMO) Task which facilitates a mechanistic investigation in mice. We found that socially-excluded mice display more severe responses to physical pain, disrupted valence encoding, and impaired neural representations of nociceptive stimuli. We performed a systematic biosensor panel and found that endocannabinoid and oxytocin signaling in the insular cortex have opposing responses during trials where mice were attending or not attending to the Social Exclusion events respectively, demonstrating distinct neuromodulatory substrates underpin different states of Social Exclusion. We also found that intra-insular blockade of oxytocin signaling increased the impact of Social Exclusion on physical pain. Together these findings suggest Social Exclusion effectively alters physical pain perception using neuromodulatory signaling in the insular cortex.
- 3 June 2025
- Published by Springer Nature in Research Square
Open AccessSocial Exclusion Amplifies Behavioral Responses to Physical Pain via Insular Neuromodulation
The “Pain Overlap Theory”1 proposes that the experience of social pain overlaps with and amplifies the experience of physical pain by sharing parts of the same underlying processing systems 2–6. In humans, the insular cortex has been implicated in this overlap of physical and social pain, but a mechanistic link has not been made2,4,5,7–9. To determine whether social pain can subsequently impact responses to nociceptive stimuli via convergent electrical signals (spikes) or convergent chemical signals (neuromodulators), we designed a novel Social Exclusion paradigm termed the Fear of Missing Out (FOMO) Task which facilitates a mechanistic investigation in mice. We found that socially-excluded mice display more severe responses to physical pain, disrupted valence encoding, and impaired neural representations of nociceptive stimuli. We performed a systematic biosensor panel and found that endocannabinoid and oxytocin signaling in the insular cortex have opposing responses during trials where mice were attending or not attending to the Social Exclusion events respectively, demonstrating distinct neuromodulatory substrates underpin different states of Social Exclusion. We also found that intra-insular blockade of oxytocin signaling increased the impact of Social Exclusion on physical pain. Together these findings suggest Social Exclusion effectively alters physical pain perception using neuromodulatory signaling in the insular cortex.
- Open Access
- 20 March 2025
- Published by Springer Nature in Research Square
Predicting Future Development of Stress-Induced Anhedonia From Cortical Dynamics and Facial Expression
The current state of mental health treatment for individuals diagnosed with major depressive disorder leaves billions of individuals with first-line therapies that are ineffective or burdened with undesirable side effects. One major obstacle is that distinct pathologies may currently be diagnosed as the same disease and prescribed the same treatments. The key to developing antidepressants with ubiquitous efficacy is to first identify a strategy to differentiate between heterogeneous conditions. Major depression is characterized by hallmark features such as anhedonia and a loss of motivation1,2, and it has been recognized that even among inbred mice raised under identical housing conditions, we observe heterogeneity in their susceptibility and resilience to stress3. Anhedonia, a condition identified in multiple neuropsychiatric disorders, is described as the inability to experience pleasure and is linked to anomalous medial prefrontal cortex (mPFC) activity4. The mPFC is responsible for higher order functions5–8, such as valence encoding; however, it remains unknown how mPFC valence-specific neuronal population activity is affected during anhedonic conditions. To test this, we implemented the unpredictable chronic mild stress (CMS) protocol9–11 in mice and examined hedonic behaviors following stress and ketamine treatment. We used unsupervised clustering to delineate individual variability in hedonic behavior in response to stress. We then performed in vivo 2-photon calcium imaging to longitudinally track mPFC valence-specific neuronal population dynamics during a Pavlovian discrimination task. Chronic mild stress mice exhibited a blunted effect in the ratio of mPFC neural population responses to rewards relative to punishments after stress that rebounds following ketamine treatment. Also, a linear classifier revealed that we can decode susceptibility to chronic mild stress based on mPFC valence-encoding properties prior to stress-exposure and behavioral expression of susceptibility. Lastly, we utilized markerless pose tracking computer vision tools to predict whether a mouse would become resilient or susceptible based on facial expressions during a Pavlovian discrimination task. These results indicate that mPFC valence encoding properties and behavior are predictive of anhedonic states. Altogether, these experiments point to the need for increased granularity in the measurement of both behavior and neural activity, as these factors can predict the predisposition to stress-induced anhedonia.
- 20 March 2025
- Published by Springer Nature in Research Square
Open AccessPredicting Future Development of Stress-Induced Anhedonia From Cortical Dynamics and Facial Expression
The current state of mental health treatment for individuals diagnosed with major depressive disorder leaves billions of individuals with first-line therapies that are ineffective or burdened with undesirable side effects. One major obstacle is that distinct pathologies may currently be diagnosed as the same disease and prescribed the same treatments. The key to developing antidepressants with ubiquitous efficacy is to first identify a strategy to differentiate between heterogeneous conditions. Major depression is characterized by hallmark features such as anhedonia and a loss of motivation1,2, and it has been recognized that even among inbred mice raised under identical housing conditions, we observe heterogeneity in their susceptibility and resilience to stress3. Anhedonia, a condition identified in multiple neuropsychiatric disorders, is described as the inability to experience pleasure and is linked to anomalous medial prefrontal cortex (mPFC) activity4. The mPFC is responsible for higher order functions5–8, such as valence encoding; however, it remains unknown how mPFC valence-specific neuronal population activity is affected during anhedonic conditions. To test this, we implemented the unpredictable chronic mild stress (CMS) protocol9–11 in mice and examined hedonic behaviors following stress and ketamine treatment. We used unsupervised clustering to delineate individual variability in hedonic behavior in response to stress. We then performed in vivo 2-photon calcium imaging to longitudinally track mPFC valence-specific neuronal population dynamics during a Pavlovian discrimination task. Chronic mild stress mice exhibited a blunted effect in the ratio of mPFC neural population responses to rewards relative to punishments after stress that rebounds following ketamine treatment. Also, a linear classifier revealed that we can decode susceptibility to chronic mild stress based on mPFC valence-encoding properties prior to stress-exposure and behavioral expression of susceptibility. Lastly, we utilized markerless pose tracking computer vision tools to predict whether a mouse would become resilient or susceptible based on facial expressions during a Pavlovian discrimination task. These results indicate that mPFC valence encoding properties and behavior are predictive of anhedonic states. Altogether, these experiments point to the need for increased granularity in the measurement of both behavior and neural activity, as these factors can predict the predisposition to stress-induced anhedonia.
- Open Access
Predicting Future Development of Stress-Induced Anhedonia From Cortical Dynamics and Facial Expression
The current state of mental health treatment for individuals diagnosed with major depressive disorder leaves billions of individuals with first-line therapies that are ineffective or burdened with undesirable side effects. One major obstacle is that distinct pathologies may currently be diagnosed as the same disease and prescribed the same treatments. The key to developing antidepressants with ubiquitous efficacy is to first identify a strategy to differentiate between heterogeneous conditions. Major depression is characterized by hallmark features such as anhedonia and a loss of motivation1,2, and it has been recognized that even among inbred mice raised under identical housing conditions, we observe heterogeneity in their susceptibility and resilience to stress3. Anhedonia, a condition identified in multiple neuropsychiatric disorders, is described as the inability to experience pleasure and is linked to anomalous medial prefrontal cortex (mPFC) activity4. The mPFC is responsible for higher order functions5–8, such as valence encoding; however, it remains unknown how mPFC valence-specific neuronal population activity is affected during anhedonic conditions. To test this, we implemented the unpredictable chronic mild stress (CMS) protocol9–11 in mice and examined hedonic behaviors following stress and ketamine treatment. We used unsupervised clustering to delineate individual variability in hedonic behavior in response to stress. We then performed in vivo 2-photon calcium imaging to longitudinally track mPFC valence-specific neuronal population dynamics during a Pavlovian discrimination task. Chronic mild stress mice exhibited a blunted effect in the ratio of mPFC neural population responses to rewards relative to punishments after stress that rebounds following ketamine treatment. Also, a linear classifier revealed that we can decode susceptibility to chronic mild stress based on mPFC valence-encoding properties prior to stress-exposure and behavioral expression of susceptibility. Lastly, we utilized markerless pose tracking computer vision tools to predict whether a mouse would become resilient or susceptible based on facial expressions during a Pavlovian discrimination task. These results indicate that mPFC valence encoding properties and behavior are predictive of anhedonic states. Altogether, these experiments point to the need for increased granularity in the measurement of both behavior and neural activity, as these factors can predict the predisposition to stress-induced anhedonia.
Open AccessPredicting Future Development of Stress-Induced Anhedonia From Cortical Dynamics and Facial Expression
The current state of mental health treatment for individuals diagnosed with major depressive disorder leaves billions of individuals with first-line therapies that are ineffective or burdened with undesirable side effects. One major obstacle is that distinct pathologies may currently be diagnosed as the same disease and prescribed the same treatments. The key to developing antidepressants with ubiquitous efficacy is to first identify a strategy to differentiate between heterogeneous conditions. Major depression is characterized by hallmark features such as anhedonia and a loss of motivation1,2, and it has been recognized that even among inbred mice raised under identical housing conditions, we observe heterogeneity in their susceptibility and resilience to stress3. Anhedonia, a condition identified in multiple neuropsychiatric disorders, is described as the inability to experience pleasure and is linked to anomalous medial prefrontal cortex (mPFC) activity4. The mPFC is responsible for higher order functions5–8, such as valence encoding; however, it remains unknown how mPFC valence-specific neuronal population activity is affected during anhedonic conditions. To test this, we implemented the unpredictable chronic mild stress (CMS) protocol9–11 in mice and examined hedonic behaviors following stress and ketamine treatment. We used unsupervised clustering to delineate individual variability in hedonic behavior in response to stress. We then performed in vivo 2-photon calcium imaging to longitudinally track mPFC valence-specific neuronal population dynamics during a Pavlovian discrimination task. Chronic mild stress mice exhibited a blunted effect in the ratio of mPFC neural population responses to rewards relative to punishments after stress that rebounds following ketamine treatment. Also, a linear classifier revealed that we can decode susceptibility to chronic mild stress based on mPFC valence-encoding properties prior to stress-exposure and behavioral expression of susceptibility. Lastly, we utilized markerless pose tracking computer vision tools to predict whether a mouse would become resilient or susceptible based on facial expressions during a Pavlovian discrimination task. These results indicate that mPFC valence encoding properties and behavior are predictive of anhedonic states. Altogether, these experiments point to the need for increased granularity in the measurement of both behavior and neural activity, as these factors can predict the predisposition to stress-induced anhedonia.
- Top Cited PapersOpen Access
- 12 July 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (7) , 1329-1339
Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules (‘syllables’) from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.
- 12 July 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (7) , 1329-1339
Top Cited PapersOpen AccessKeypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules (‘syllables’) from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.