MSMelanie Segado

Melanie Segado

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Melanie Segado has a total of 43 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A preregistered, open pipeline for early cerebral palsy risk assessment from infant videos

    by Melanie Segado, Laura A Prosser, Andrea F Duncan, Michelle J Johnson, Konrad P Kording

    Cerebral palsy (CP), affecting approximately 1 in 500 children due to abnormal brain development, impacts movement control. Early risk assessment via the general movements assessment (GMA) at 3–4 months is highly predictive for CP but relies on trained clinicians. Machine-learning-based approaches for predicting GMA score from video have shown considerable promise, but typically rely on dataset-specific preprocessing, custom feature sets, and manually designed model pipelines, which make external benchmarking more difficult. This, combined with strict privacy constraints on sharing data, makes it challenging to train and evaluate models across datasets, which is important for assessing clinical utility. There is therefore a need to develop approaches that will work across different datasets to enable multi-site dataset aggregation and model training. To address this gap, we developed an end-to-end pipeline that uses off-the-shelf pose estimation, general-purpose feature extraction, and automated machine learning—none of which are tuned to a specific dataset. We applied this approach to a newly generated large dataset of 1053 infants (with approximately 10–12% positive class for adverse GMA outcome, drawn from a high-risk clinical cohort) within a preregistered study design. Model performance was evaluated on a strict “lock-box” test set, which remained untouched during any phase of model development or preprocessing optimization, and only used for evaluation once the final model and pipeline had been preregistered. The developed model achieved moderate predictive accuracy for clinician-assessed GMA scores (area under the receiver operating characteristic curve, ROC-AUC = 0.77; area under the precision-recall curve, PR-AUC = 0.41). The moderate accuracy is noteworthy given the 10–12% positive class prevalence, and power-law scaling of ROC-AUC as a function of increasing dataset size. By releasing de-identified feature data and open-source code, and simplifying the training pipeline using AutoML, our work establishes essential groundwork for future robust, globally relevant CP screening tools suitable for low-resource settings.


  • Open Access

    PrimateFace: A Machine Learning Resource for Automated Face Analysis in Human and Non-human Primates

    by Felipe Parodi, Jordan K. Matelsky, Alessandro P. Lamacchia, Melanie Segado, Yaoguang Jiang, Alejandra Regla-Vargas, Liala Sofi, Clare Kimock, Bridget M. Waller, Michael L. Platt, et al.

    Machine learning has revolutionized human face analysis, but equivalent tools for non-human primates remain limited and species-specific, hindering progress in neuroscience, anthropology, and conservation. Here, we present PrimateFace, a comprehensive, cross-species platform for primate facial analysis comprising a systematically curated dataset of 260,000+ images spanning over 60 genera, including a genus-balanced subset of 60,000 images, annotated with bounding boxes and facial landmark configurations. Face detection and facial landmark estimation models trained on PrimateFace achieve high cross-species performance, from tarsiers to gorillas, achieving performance comparable to baseline models trained exclusively on human data (0.34 vs. 0.39 mAP for face detection; 0.061 vs. 0.053 normalized landmark error), demonstrating the generalization benefits of cross-species training. PrimateFace enables diverse downstream applications including individual recognition, gaze analysis, and automated extraction of stereotyped (e.g., lip-smacking) and subtle (e.g., soft left turn) facial movements. PrimateFace provides a standardized platform for facial phenotyping across the primate order, empowering data-driven studies that advance the health and well-being of human and non-human primates. All models, notebooks, and data can be found at github.com/KordingLab/PrimateFace .


  • Open Access

    Grounding Intelligence in Movement

    by Melanie Segado, Felipe Parodi, Jordan K. Matelsky, Michael L. Platt, Eva B. Dyer, Konrad P. Kording

    Recent advances in machine learning have dramatically improved our ability to model language, vision, and other high-dimensional data, yet they continue to struggle with one of the most fundamental aspects of biological systems: movement. Across neuroscience, medicine, robotics, and ethology, movement is essential for interpreting behavior, predicting intent, and enabling interaction. Despite its core significance in our intelligence, movement is often treated as an afterthought rather than as a rich and structured modality in its own right. This reflects a deeper fragmentation in how movement data is collected and modeled, often constrained by task-specific goals and domain-specific assumptions. But movement is not domain-bound. It reflects shared physical constraints, conserved morphological structures, and purposeful dynamics that cut across species and settings. We argue that movement should be treated as a primary modeling target for AI. It is inherently structured and grounded in embodiment and physics. This structure, often allowing for compact, lower-dimensional representations (e.g., pose), makes it more interpretable and computationally tractable to model than raw, high-dimensional sensory inputs. Developing models that can learn from and generalize across diverse movement data will not only advance core capabilities in generative modeling and control, but also create a shared foundation for understanding behavior across biological and artificial systems. Movement is not just an outcome, it is a window into how intelligent systems engage with the world.


  • Open Access

    Considering Theory-Based Gamification in the Co-Design and Development of a Virtual Reality Cognitive Remediation Intervention for Depression (bWell-D): Mixed Methods Study

    by Mark Hewko, Vincent Gagnon Shaigetz, Michael S Smith, Elicia Kohlenberg, Pooria Ahmadi, Maria Elena Hernandez Hernandez, Catherine Proulx, Anne Cabral, Melanie Segado, Trisha Chakrabarty, et al.

    Background: In collaboration with clinical domain experts, we developed a prototype of immersive virtual reality (VR) cognitive remediation for major depressive disorder (bWell-D). In the development of a new digital intervention, there is a need to determine the effective components and clinical relevance using systematic methodologies. From an implementation perspective, the effectiveness of digital intervention delivery is challenged by low uptake and high noncompliance rates. Gamification may play a role in addressing this as it can boost adherence. However, careful consideration is required in its application to promote user motivation intrinsically. Objective: We aimed to address these challenges through an iterative process for development that involves co-design for developing content as well as in the application of gamification while also taking into consideration behavior change theories. This effort followed the methodological framework guidelines outlined by an international working group for development of VR therapies. Methods: In previously reported work, we collected qualitative data from patients and care providers to understand end-user perceptions on the use of VR technologies for cognitive remediation, reveal insights on the drivers for behavior change, and obtain suggestions for changes specific to the VR program. In this study, we translated these findings into concrete representative software functionalities or features and evaluated them against behavioral theories to characterize gamification elements in terms of factors that drive behavior change and intrinsic engagement, which is of particular importance in the context of cognitive remediation. The implemented changes were formally evaluated through user trials. Results: The results indicated that feedback from end users centered on using gamification to add artificial challenges, personalization and customization options, and artificial assistance while focusing on capability as the behavior change driver. It was also found that, in terms of promoting intrinsic engagement, the need to meet competence was most frequently raised. In user trials, bWell-D was well tolerated, and preliminary results suggested an increase in user experience ratings with high engagement reported throughout a 4-week training program. Conclusions: In this paper, we present a process for the application of gamification that includes characterizing what was applied in a standardized way and identifying the underlying mechanisms that are targeted. Typical gamification elements such as points and scoring and rewards and prizes target motivation in an extrinsic fashion. In this work, it was found that modifications suggested by end users resulted in the inclusion of gamification elements less commonly observed and that tend to focus more on individual ability. It was found that the incorporation of end-user feedback can lead to the application of gamification in broader ways, with the identification of elements that are potentially better suited for mental health domains.


  • Open Access

    An Integrated Platform Combining Immersive Virtual Reality and Physiological Sensors for Systematic and Individualized Assessment of Stress Response (bWell): Design and Implementation Study

    by Budhachandra Khundrakpam, Melanie Segado, Jesse Pazdera, Vincent Gagnon Shaigetz, Joshua A Granek, Nusrat Choudhury

    Background: Stress is a pervasive issue in modern society, manifesting in various forms such as emotional, physical, and work-related stress, each with distinct impacts on individuals and society. Traditional stress studies often rely on psychological, performance, or social tests; however, recently, immersive virtual reality (VR), which provides a sense of presence and natural interaction, offers the opportunity to simulate real-world tasks and stressors in controlled environments. Despite its potential, the use of VR to investigate the multifaceted manifestations of stress has not been thoroughly explored. Objective: This study aimed to explore the feasibility of using a VR-based platform, bWell, to elicit multifaceted stress responses and measure the resulting behavioral and physiological changes. Specifically, we aimed to design various VR stress exercises based on neurocardiac models to systematically test cardiac functioning within specific contexts of self-regulation (executive functioning, physical efforts, and emotional regulation). Methods: The development process adhered to guidelines for VR clinical trials and complex health interventions, encompassing 3 phases: preparation, development, and verification. The preparation phase involved a comprehensive literature review to establish links between stress, the heart, and the brain, leading to the formulation of a conceptual model based on the Neurovisceral Integration Model (NVIM) and Vagal Tank Theory (VTT). The development phase involved designing VR exercises targeting specific stressors and integrating physiological sensors such as photoplethysmography (PPG) and electromyography (EMG) to capture heart rate variability (HRV) and facial expressions. The verification phase, conducted with a small number of trials, aimed to design a study and implement a workflow for testing the feasibility, acceptability, and tolerability of the VR exercises. In addition, the potential for capturing physiological measures along with subjective ratings of stress for specific dimensions was assessed. Results: Verification trials demonstrated that the VR exercises were well tolerated, with negligible cybersickness and high user engagement. The different VR exercises successfully elicited the intended stress demands, along with the physiological responses. Conclusions: The study presents a novel VR-based experimental setup that allows a systematic and individualized assessment of stress responses, paving the way for future research to identify features that confer stress resilience and help individuals manage stress effectively. While our conceptual model highlights the role of HRV in providing valuable insights into stress responses, future research will involve multivariate and machine learning analyses to predict individual stress responses based on comprehensive sensor data, including EMG and the VR-based behavioral data, ultimately guiding personalized stress management interventions.


  • Open Access

    Data-Driven Early Prediction of Cerebral Palsy Using AutoML and interpretable kinematic features

    by Melanie Segado, Laura Prosser, Andrea F. Duncan, Michelle J. Johnson, Konrad P. Kording

    Early identification of cerebral palsy (CP) remains a major challenge due to the reliance on expert assessments that are time-intensive and not scalable. Consequently, a range of studies have aimed at using machine learning to predict CP scores based on motion tracking,e.g. from video data. These studies generally predict clinical scores which are a proxy for CP risk. However, clinicians do not REALLY want to estimate scores, they want to estimate the patients’ risk of developing clinical symptoms. Here we present a data-driven machine-learning (ML) pipeline that extracts movement features from infant video based motion tracking and estimates CP risk using AutoML. Using AutoSklearn, our framework minimizes risk of overfitting by abstracting away researcher-driver hyperparameter optimization. Trained on movement data from 3- to 4-month-old infants, our classifier predicts a highly indicative clinical score (General Movements Assessment [GMA]) with an ROC-AUC of 0.78 on a held-out test set, indicating that kinematic movement features capture clinically relevant variability. Without retraining, the same model predicts the risk of cerebral palsy outcomes at later clinical follow-ups with an ROC-AUC of 0.74, demonstrating that early motor representations generalize to long-term neurodevelopmental risk. We employ pre-registered lock-box validation to ensure rig-orous performance evaluation. This study highlights the potential of AutoML-powered movement analytics for neurodevelopmental screening, demonstrating that data-driven feature extraction from movement trajectories can provide an interpretable and scalable approach to early risk assessment. By integrating pre-trained vision transformers, AutoML-driven model selection, and rigorous validation protocols, this work advances the use of video-derived movement features for scalable, data-driven clinical assessment, demonstrating how computational methods based on readily available data like infant videos can enhance early risk detection in neurodevelopmental disorders. CCS Concepts: Computing methodologies → Machine learning approaches ; Applied computing → Health informatics . ACM Reference Format: Melanie Segado, Laura Prosser, Andrea F. Duncan, Michelle J. Johnson, and Konrad P. Kording.. Data-Driven Early Prediction of Cerebral Palsy Using AutoML and interpretable kinematic features. In. ACM, New York, NY, USA, 8 pages.


  • Open Access

    A preregistered, Open Pipeline for Early Cerebral Palsy Risk Assessment from Infant Videos

    by Melanie Segado, Laura A. Prosser, Andrea F. Duncan, Michelle J. Johnson, Konrad P. Kording

    Cerebral Palsy (CP), affecting approximately 1 in 500 children due to abnormal brain development, impacts movement control. Early risk assessment via the General Movements Assessment (GMA) at 3-4 months is highly predictive for CP but relies on trained clinicians. Machine-learning-based approaches for predicting GMA score from video have shown considerable promise, but are not openly available and rely on fine-tuned pre-processing steps, hand-crafted feature sets, and experimenter-driven hyperparameter selection. This, combined with strict privacy constraints on sharing data, limits the extent to which models can be trained and tested across datasets, thus reducing clinical impact. There is therefore a need to develop approaches that will work across different datasets to enable multi-site dataset aggregation and model training. To address this gap, we developed an end-to-end pipeline that uses off-the-shelf pose estimation, general-purpose feature extraction, and automated machine learning—none of which are tuned to a specific dataset. We applied this approach to a newly generated large dataset of 1063 infants (with approximately 12% positive class for adverse GMA outcome, drawn from a high-risk clinical cohort) within a preregistered study design. Model performance was evaluated on a strict “lock-box” validation set, which remained untouched during any phase of model development or pre-processing optimization. The developed model achieved moderate predictive accuracy for clinician-assessed GMA scores (Area Under the Receiver Operating Characteristic Curve, ROC-AUC = 0.79; Area Under the Precision-Recall Curve, PR-AUC = 0.34). The moderate accuracy is noteworthy given the 12% positive class prevalence. By releasing de-identified feature data and open-source code, and simplifying the training pipeline using automated machine learning, our work establishes essential groundwork for future robust, globally relevant CP screening tools suitable for low-resource settings. Key Points Introduced an open, accessible video-based pipeline to predict General Movements Assessment (GMA) scores (a key early indicator of Cerebral Palsy [CP] risk), providing a foundation for more scalable pediatric risk assessment. Rigorously validated this pipeline on a large infant cohort (1063 videos), employing a pre-registered design and a “lock-box” test set to ensure robust evaluation and minimize the risk of overly optimistic performance estimates. Demonstrated that relatively simple movement features, derived from hand-held camera recordings achieves moderate predictive accuracy for GMA scores (ROC-AUC 0.79, PR-AUC 0.34) even under these stringent validation conditions. Designed the pipeline to facilitate broader application and collaborative research, particularly through its use of generalizable pose estimation and by enabling the extraction and sharing of de-identified movement features for aggregated dataset creation across clinical sites. Released the entire pipeline as open-source (including data processing, feature computation, and AutoML components) to promote transparency, reproducibility, and empower the research community to collaboratively build and refine more impactful early CP risk assessment tools.


  • Open Access

    An Integrated Platform Combining Immersive Virtual Reality and Physiological Sensors for Systematic and Individualized Assessment of Stress Response (bWell): Design and Implementation Study (Preprint)

    by Budhachandra Khundrakpam, Melanie Segado, Jesse Pazdera, Vincent Gagnon Shaigetz, Joshua A Granek, Nusrat Choudhury

    BACKGROUND Stress is a pervasive issue in modern society, manifesting in various forms such as emotional, physical, and work-related stress, each with distinct impacts on individuals and society. Traditional stress studies often rely on psychological, performance, or social tests; however, recently, immersive virtual reality (VR), which provides a sense of presence and natural interaction, offers the opportunity to simulate real-world tasks and stressors in controlled environments. Despite its potential, the use of VR to investigate the multifaceted manifestations of stress has not been thoroughly explored. OBJECTIVE This study aimed to explore the feasibility of using a VR-based platform, bWell, to elicit multifaceted stress responses and measure the resulting behavioral and physiological changes. Specifically, we aimed to design various VR stress exercises based on neurocardiac models to systematically test cardiac functioning within specific contexts of self-regulation (executive functioning, physical efforts, and emotional regulation). METHODS The development process adhered to guidelines for VR clinical trials and complex health interventions, encompassing 3 phases: preparation, development, and verification. The preparation phase involved a comprehensive literature review to establish links between stress, the heart, and the brain, leading to the formulation of a conceptual model based on the Neurovisceral Integration Model (NVIM) and Vagal Tank Theory (VTT). The development phase involved designing VR exercises targeting specific stressors and integrating physiological sensors such as photoplethysmography (PPG) and electromyography (EMG) to capture heart rate variability (HRV) and facial expressions. The verification phase, conducted with a small number of trials, aimed to design a study and implement a workflow for testing the feasibility, acceptability, and tolerability of the VR exercises. In addition, the potential for capturing physiological measures along with subjective ratings of stress for specific dimensions was assessed. RESULTS Verification trials demonstrated that the VR exercises were well tolerated, with negligible cybersickness and high user engagement. The different VR exercises successfully elicited the intended stress demands, along with the physiological responses. CONCLUSIONS The study presents a novel VR-based experimental setup that allows a systematic and individualized assessment of stress responses, paving the way for future research to identify features that confer stress resilience and help individuals manage stress effectively. While our conceptual model highlights the role of HRV in providing valuable insights into stress responses, future research will involve multivariate and machine learning analyses to predict individual stress responses based on comprehensive sensor data, including EMG and the VR-based behavioral data, ultimately guiding personalized stress management interventions.


  • Open Access

    Considering Theory-Based Gamification in the Co-Design and Development of a Virtual Reality Cognitive Remediation Intervention for Depression (bWell-D): Mixed Methods Study (Preprint)

    by Mark Hewko, Vincent Gagnon Shaigetz, Michael S Smith, Elicia Kohlenberg, Pooria Ahmadi, Maria Elena Hernandez Hernandez, Catherine Proulx, Anne Cabral, Melanie Segado, Trisha Chakrabarty, et al.

    BACKGROUND In collaboration with clinical domain experts, we developed a prototype of immersive virtual reality (VR) cognitive remediation for major depressive disorder (bWell-D). In the development of a new digital intervention, there is a need to determine the effective components and clinical relevance using systematic methodologies. From an implementation perspective, the effectiveness of digital intervention delivery is challenged by low uptake and high noncompliance rates. Gamification may play a role in addressing this as it can boost adherence. However, careful consideration is required in its application to promote user motivation intrinsically. OBJECTIVE We aimed to address these challenges through an iterative process for development that involves co-design for developing content as well as in the application of gamification while also taking into consideration behavior change theories. This effort followed the methodological framework guidelines outlined by an international working group for development of VR therapies. METHODS In previously reported work, we collected qualitative data from patients and care providers to understand end-user perceptions on the use of VR technologies for cognitive remediation, reveal insights on the drivers for behavior change, and obtain suggestions for changes specific to the VR program. In this study, we translated these findings into concrete representative software functionalities or features and evaluated them against behavioral theories to characterize gamification elements in terms of factors that drive behavior change and intrinsic engagement, which is of particular importance in the context of cognitive remediation. The implemented changes were formally evaluated through user trials. RESULTS The results indicated that feedback from end users centered on using gamification to add artificial challenges, personalization and customization options, and artificial assistance while focusing on capability as the behavior change driver. It was also found that, in terms of promoting intrinsic engagement, the need to meet competence was most frequently raised. In user trials, bWell-D was well tolerated, and preliminary results suggested an increase in user experience ratings with high engagement reported throughout a 4-week training program. CONCLUSIONS In this paper, we present a process for the application of gamification that includes characterizing what was applied in a standardized way and identifying the underlying mechanisms that are targeted. Typical gamification elements such as points and scoring and rewards and prizes target motivation in an extrinsic fashion. In this work, it was found that modifications suggested by end users resulted in the inclusion of gamification elements less commonly observed and that tend to focus more on individual ability. It was found that the incorporation of end-user feedback can lead to the application of gamification in broader ways, with the identification of elements that are potentially better suited for mental health domains.

  • Open Access

    The influence of auditory change on cognitive processes associated with aging in virtual reality

    by Farooq Kamal, Melanie Segado, Maxime Perron, Brian Lau, Vincent Gagnon Shaigetz, Claude Alain, Nusrat Choudhury

    Background: With increased age, older adults experience subtle changes in their cognitive functioning. For example, it is well established that our ability to complete dual‐tasking activities decreases with increased age and cognitive decline. One issue with current dual‐task paradigms is the lack of ecological value in lab‐based environments. The present study employed working memory (WM) tasks designed within an immersive and interactive virtual reality (VR) environment by the National Research Council Canada, to examine the processing of unattended input using event‐related potentials (ERPs) which measure the changes in the electrical activity of the brain.Method: Twenty young adults (ages 17‐30) and twenty healthy older adults (ages 65+) were recruited from the Rotman Research Institute. Participants were engaged in a VR‐WM task while to‐be‐ignored auditory stimuli were presented. The difficulty of the VR tasks was designed to be either very easy or very difficult. Auditory stimulus with an average intensity of 60‐90 dB SPL were presented in a fast (low saliency) and a slow (high saliency) presentation rate.Result: During the VR task, young adults’ accuracy was significantly higher in the easy condition than in the difficult condition (p < .001). When auditory stimuli were presented rapidly, a robust ERP was elicited at the frontal (p = 0.005) and central (p = 0.003) brain regions. The slowly presented auditory stimulus elicited a robust ERP at the central brain region (p = 0.003). Preliminary data indicates that accuracy was lower in older adults than young adults for both conditions. Older adults also had lower accuracy in the hard condition than in the easy condition.Conclusion: The ERPs examined here show that participants were able to successfully switch processing priorities away from the VR‐WM task to salient auditory stimulus, but that older adults were less efficient than younger adults. These tools (VR, auditory stimuli, and ERPs) may be useful at detecting subtle electrophysiological and cognitive changes that occur with age and cognitive decline. They may offer promise to identify the pathology underlying cognitive impairment in aging and could reveal the electrophysiological changes associated with conversion to dementia at an individual level in clinical and non‐clinical settings.