KRKaren Rosero

Karen Rosero

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Karen Rosero has a total of 23 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Multimodal Biomarkers of Dysarthria: Severity-Conditioned Contrastive Alignment of Speech, Text, and Facial Asymmetry

    by Karen Rosero, Chi-Chun Lee, Rami R. Hallac, Carlos Busso

    Dysarthria is a motor speech disorder that affects speech production, and severity assessment can benefit from combining acoustic and visual biomarkers. While recent advances have improved unimodal-based evaluations, most systems do not fully leverage facial motion, asymmetry, or the relationship between the produced speech and its intended linguistic content. In this work, we propose a framework for MultiModal Dysarthria Severity assessment (MMDyS), which exploits the alignment of speech, text, and visual biomarkers. We introduce a severity-conditioned contrastive objective that treats speech–text mismatch as a continuous signal of impairment, while the visual modality relies on hemiface lateralization to capture asymmetric articulatory patterns, complemented by temporal segmentation and flow motion descriptors. These representations are integrated through a mixture-of-experts mechanism with cross-modal alignment, enabling adaptive weighting of speech, text, and visual evidence. Evaluated on the Mandarin speech dysarthria multimodal (MSDM) benchmark, MMDyS achieves a subject-level macro F1 of 0.92 and the composite multimodal dysarthria severity assessment (MDSA) score of 11.7% over the current state-of-the-art. Our results show that modeling speech–text divergence and facial asymmetry leads to more informative and robust multimodal representations for dysarthria severity assessment. Code is available at: https://github.com/Lab-MSP/MMDyS.

  • Open Access

    DiffusionCleft: Facial Anomaly Synthesis Guided by Text

    by Karen Rosero, Lucas M Harrison, Alex A Kane, Rami R Hallac, Carlos Busso

  • Open Access

    Deep Learning-Based Assessment of Lip Symmetry for Patients With Repaired Cleft Lip

    by Karen Rosero, Ali N. Salman, Lucas M. Harrison, Alex A. Kane, Carlos Busso, Rami R. Hallac

    Objective: Post-surgical lip symmetry assessment is a key indicator of cleft repair success. Traditional methods rely on distances between anatomical landmarks, which are impractical for video analysis and overlook texture and appearance. We propose an artificial intelligence (AI) approach to automate this process, analyzing lateral lip morphology for a quantitative symmetry evaluation. Design: We utilize contrastive learning to quantify lip symmetry by measuring the similarity between the representations of the sides, which is subsequently used to classify the severity of asymmetry. Our model does not require patient images for training. Instead, we introduce dissimilarities in face images from open datasets using two methods: temporal misalignment for video frames and face transformations to simulate lip asymmetry observed in the target population. The model differentiates the left and right image representations to assess asymmetry. We evaluated our model on 146 images of patients with repaired cleft lip. Results: The deep learning model trained with face transformations categorized patient images into five asymmetry levels, achieving a weighted accuracy of 75% and a Pearson correlation of 0.31 with medical expert human evaluations. The model utilizing temporal misalignment achieved a weighted accuracy of 69% and a Pearson correlation of 0.27 for the same classification task. Conclusions: We propose an automated approach for assessing lip asymmetry in patients with repaired cleft lip by transforming facial images of control subjects to train a deep learning model, eliminating manual anatomical landmarks. Our promising results provide a more efficient and objective tool for evaluating surgical outcomes.


  • Open Access

    w2v-SELD: A Sound Event Localization and Detection Framework for Self-Supervised Spatial Audio Pre-Training

    by Orlem Lima Dos Santos, Karen Rosero, Bruno Masiero, Roberto de Alencar Lotufo

    Sound Event Localization and Detection (SELD) is a critical challenge in various industrial applications, such as autonomous systems, smart cities, and audio surveillance, which require accurate identification and localization of sound events in complex environments. Traditional supervised approaches heavily rely on large, annotated multichannel audio datasets, which are expensive and time-consuming to produce. This paper addresses this limitation by introducing the w2v-SELD architecture, a self-supervised model adapted from the wav2vec 2.0 framework to learn effective sound event representations directly from raw, unlabeled 3D audio data. The proposed model follows a two-stage process: pre-training on large, unlabeled 3D audio datasets to capture high-level features, followed by fine-tuning with a smaller, labeled SELD dataset. Experimental results show that our w2v-SELD method outperforms baseline models on Detection and Classification of Acoustic Scenes and Events (DCASE) challenges, achieving a 66% improvement for DCASE TAU-2019 and a 57% improvement on DCASE TAU-2020 with respect to baseline systems. The w2v-SELD model performs competitively with state-of-the-art supervised methods, highlighting its potential to significantly reduce the dependency on labeled data in industrial SELD applications. The code and pre-trained parameters of our w2v-SELD model are available in this repository.


  • Open Access

    Lip Abnormality Detection for Patients with Repaired Cleft Lip and Palate: A Lip Normalization Approach

    by Karen Rosero, Ali N. Salman, Rami R. Hallac, Carlos Busso

    The cleft lip condition arises from the incomplete fusion of oral and labial structures during fetal development, impacting vital functions. After surgical closure, patients commonly present with abnormal lip shape, which may require secondary revision surgery for both aesthetic and functional improvement. However, a lack of standardized evaluation methods complicates decision-making for secondary surgery. To address this limitation, we propose a transformer-based lip normalization approach that filters out abnormalities and achieves a standardized appearance while preserving individual anatomy. An innovation of our approach is a lip transformation method using available face datasets to mimic repaired cleft lip shapes, enabling the training of deep learning models without using patients’ data. We employ a Siamese convolutional neural network that processes pre- and post-normalization images to detect lip abnormalities with an accuracy of 89%. We compare our approach with a single-branch model without lip normalization, which reached an accuracy of 60%. Our approach has the potential to provide an impartial view to determine the need for revision surgery while also assisting in the selection of healthcare tools specialized for patients with repaired cleft lip. The code for this work is available in our anonymous repository 1.


  • Open Access

    A-MAD: An Adaptive-Microseismic Activity Detector Based on Gaussian Mixture Models and Spectral Subtraction

    by Karen Rosero, Carla Parra, Felipe Grijalva, Julio César Larco, Román Lara-Cueva, Nathaly Orozco Garzón

    Continuous monitoring of active volcanoes is essential for understanding their behavior and providing accurate forecasts and warnings. Automating the detection of volcano microseismic events plays a crucial role in large-scale analysis, early warning systems, and efficient handling of large-scale data. This paper presents a novel approach for the automated detection of microseismic events associated with volcanic activity, focusing on the Cotopaxi volcano in Ecuador. Inspired by voice activity detection (VAD) systems used in speech processing, we propose an Adaptive-Microseismic Activity Detector (A-MAD) that modifies VAD techniques to identify frames within seismic signals containing volcanic activity. The A-MAD system incorporates a spectral subtraction stage to mitigate the impact of environmental noise and employs Gaussian Mixture Models (GMMs) to model the probabilistic distribution of microseismic events. Mel Frequency Cepstral Coefficients (MFCCs) are used as features to describe the seismic signals, enabling adaptability for varying noise levels. Experimental results on signals from Cotopaxi Volcano in Ecuador demonstrate the effectiveness of the A-MAD system, achieving high accuracy (96.39% for discrete events and 98.45% for continuous signals) while meeting the efficiency requirements of volcano monitoring institutions. This work contributes to the advancement of early warning systems for volcanic eruptions, providing a robust and adaptive approach for microseismic activity detection.


  • Sound Events Localization and Detection Using Bio-Inspired Gammatone Filters and Temporal Convolutional Neural Networks

    by Karen Gissell Rosero Jacome, Felipe Leonel Grijalva, Bruno Sanches Masiero

    The auditory brain circuits are biologically constructed to recognize and localize sounds by encoding a combination of cues that help individuals interpret sounds. The development of computational methods inspired by human capacities has established opportunities for improving machine hearing. Recent studies based on deep learning show that using convolutional recurrent neural networks (CRNNs) is a promising approach for sound event detection and localization in spatial sound. Nevertheless, depending on the sound environment, the performance of these systems is still far from reaching perfect metrics. Therefore, this work intends to boost the performance of state-of-the-art (SOTA) systems by using bio-inspired gammatone auditory filters and intensity vectors (IVs) for the acoustic feature extraction stage, along with the implementation of a temporal convolutional network (TCN) block into a CRNN model, to capture long term dependencies. Three data augmentation techniques are applied to increase the small number of samples in spatial audio datasets. The mentioned stages constitute our proposed Gammatone-based Sound Events Localization and Detection (G-SELD) system, which exceeded the SOTA results on four spatial audio datasets with different levels of acoustical complexity and with up to three sound sources overlapping in time.


  • Open Access

    Song Emotion Recognition: a Performance Comparison Between Audio Features and Artificial Neural Networks

    by Pedro Benevenuto Valadares, Karen Gissell Rosero Jácome, Arthur Nicholas dos Santos, Bruno Sanches Masiero

    When songs are composed or performed, there is often an intent by the singer/songwriter of expressing feelings or emotions through it. For humans, matching the emotiveness in a musical composition or performance with the subjective perceptiveness of an audience can be quite challenging. Fortunately, the machine learning approach for this problem is simpler. Usually, it takes a data-set, from which audio features are extracted to present this information to a data-driven model, that will, in turn, train predicting the highest probability of an input song matching a target emotion. In this paper†, we studied the most common features and models used in recent publications to tackle this problem, revealing which ones are best suited for songs a cappella.


  • Open Access

    Sound Events Localization and Detection Using Bio-inspired Gammatone Filters and Temporal Convolutional Neural Networks

    by Karen Rosero, Felipe Grijalva, Bruno Masiero

    This manuscript addresses the problem of detecting, classifying, and localizing sound sources in an acoustic scene of spatial audio. We propose using bio-inspired Gammatone auditory filters for the acoustic feature extraction stage and a novel deep learning architecture encompassing convolutional, recurrent, and temporal convolutional blocks. Our system exceeded the state-of-the-art metrics on four spatial audio datasets with different levels of acoustical complexity and up to three sound sources overlapping in time. Furthermore, we also performed a comparative analysis of the gap between machine and human hearing, evidencing that our results have already exceeded the human performance in non-reverberant scenarios.