Hyun-Seo Shin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hyun-Seo Shin has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Chan-Yeong Lim
School of Computer Science, University of Seoul, Seoul, South Korea
5 shared publications - Ha-Jin Yu
University of Seoul
5 shared publications - Jungwoo Heo
School of Computer Science, University of Seoul, Seoul, South Korea
5 shared publications - Jisoo Son
Intelligent Robotics Laboratory, University of Seoul, south lorea, Republic of Korea
3 shared publications - Seung-Bin Kim
Intelligent Robotics Laboratory, University of Seoul, south lorea, Republic of Korea
3 shared publications - Ju-Ho Kim
School of Computer Science, University of Seoul, Seoul, South Korea
2 shared publications
Latest Publications
- Open Access
- 2 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 54179-54191
Comprehensive Validation of Bridge Module and EBM Loss for One-Class Audio Deepfake Detection
In audio deepfake detection, a one-class perspective is emerging as a promising route to improved generalization. Following this view, we previously proposed a framework that employs an enhanced bona fide modeling (EBM) loss. The EBM loss is designed to encourage bona fide utterances to form a compact and coherent normal region in the embedding space, while explicitly pushing deepfake utterances away from this region. In other words, it systematically reshapes the decision boundary around the distribution of bona fide speech so that a one-class decision criterion can generalize to unseen attack types. However, the prior study did not systematically examine training-related factors (e.g., augmentation policy and input utterance length) or model dependence across diverse frontends and backbones. Accordingly, this paper conducts a focused validation along four axes: (i) data augmentation, (ii) hyperparameters of loss function, (iii) train-test utterance length, and (iv) portability across several frontends and backbones. Through experiments conducted on the ASVspoof 2019 LA, 2021 LA, and 2021 DF datasets, we confirm that the EBM framework consistently improves performance across diverse architecture combinations and provide concise guidelines for robust deployment under distribution shift.
- 2 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 54179-54191
Open AccessComprehensive Validation of Bridge Module and EBM Loss for One-Class Audio Deepfake Detection
In audio deepfake detection, a one-class perspective is emerging as a promising route to improved generalization. Following this view, we previously proposed a framework that employs an enhanced bona fide modeling (EBM) loss. The EBM loss is designed to encourage bona fide utterances to form a compact and coherent normal region in the embedding space, while explicitly pushing deepfake utterances away from this region. In other words, it systematically reshapes the decision boundary around the distribution of bona fide speech so that a one-class decision criterion can generalize to unseen attack types. However, the prior study did not systematically examine training-related factors (e.g., augmentation policy and input utterance length) or model dependence across diverse frontends and backbones. Accordingly, this paper conducts a focused validation along four axes: (i) data augmentation, (ii) hyperparameters of loss function, (iii) train-test utterance length, and (iv) portability across several frontends and backbones. Through experiments conducted on the ASVspoof 2019 LA, 2021 LA, and 2021 DF datasets, we confirm that the EBM framework consistently improves performance across diverse architecture combinations and provide concise guidelines for robust deployment under distribution shift.
- Open Access
- 25 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 35485-35494
Mu-Mixer: A Hierarchical MLP-Based Framework for Cover Song Identification
Cover Song Identification (CSI) is a fundamental task in music information retrieval, essential for copyright protection and content organization. While Convolutional Neural Networks (CNNs) and Transformers have significantly advanced CSI, they suffer from limited receptive fields and quadratic computational complexity, respectively. In this paper, we propose Mu-Mixer, a novel all-MLP architecture tailored for CSI that achieves global receptive fields with linear complexity. In contrast to image-based MLP-Mixers, which indiscriminately mix information across the temporal axis, Mu-Mixer incorporates a Multi-scale Temporal Mixing Block (MTMB). This strategy hierarchically segments the time dimension T into multiple scales to preserve local rhythmic motifs while capturing long-range structural dependencies. We evaluate Mu-Mixer on the AI-Hub Music Similarity dataset, where it achieves state-of-the-art performance. Our results demonstrate that Mu-Mixer provides a superior balance between identification accuracy and computational efficiency, proving that self-attention is not a prerequisite for modeling long-range acoustic dependencies in musical signals.
- 25 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 35485-35494
Open AccessMu-Mixer: A Hierarchical MLP-Based Framework for Cover Song Identification
Cover Song Identification (CSI) is a fundamental task in music information retrieval, essential for copyright protection and content organization. While Convolutional Neural Networks (CNNs) and Transformers have significantly advanced CSI, they suffer from limited receptive fields and quadratic computational complexity, respectively. In this paper, we propose Mu-Mixer, a novel all-MLP architecture tailored for CSI that achieves global receptive fields with linear complexity. In contrast to image-based MLP-Mixers, which indiscriminately mix information across the temporal axis, Mu-Mixer incorporates a Multi-scale Temporal Mixing Block (MTMB). This strategy hierarchically segments the time dimension T into multiple scales to preserve local rhythmic motifs while capturing long-range structural dependencies. We evaluate Mu-Mixer on the AI-Hub Music Similarity dataset, where it achieves state-of-the-art performance. Our results demonstrate that Mu-Mixer provides a superior balance between identification accuracy and computational efficiency, proving that self-attention is not a prerequisite for modeling long-range acoustic dependencies in musical signals.
- Open Access
- 30 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 190371-190378
A Speaker Verification System Based on a Modified MLP-Mixer Student Model for Transformer Compression
Speaker verification (SV) systems have recently achieved remarkable progress through self-supervised learning (SSL) models such as Wav2Vec2 and WavLM. Despite their strong performance, these Transformer-based models remain computationally intensive, limiting their applicability in mobile or real-time applications. Knowledge distillation has been employed to compress large SSL teachers into lightweight student models, yet most existing approaches retain Transformer architectures, thereby preserving the quadratic cost of attention mechanisms. In this work, we investigate MLP-Mixer–based student models as an alternative to Transformer students for SSL knowledge distillation. We propose a speech-oriented redesign of the MLP-Mixer that incorporates 1D convolution in the token-mixing stage to capture local temporal dependencies, and a Max-Feature-Map operation in the channel-mixing stage to emphasize speaker-discriminative frequency bands. Distilling knowledge from a WavLM teacher to this modified MLP-Mixer addresses the challenges of cross-architecture distillation. Extensive experiments on diverse benchmarks, including VoxCeleb, VCMix, VoxSRC, and VOiCES, demonstrate that the proposed approach achieves superior verification performance and computational efficiency compared to Transformer-based students, highlighting its potential for practical deployment.
- 30 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 190371-190378
Open AccessA Speaker Verification System Based on a Modified MLP-Mixer Student Model for Transformer Compression
Speaker verification (SV) systems have recently achieved remarkable progress through self-supervised learning (SSL) models such as Wav2Vec2 and WavLM. Despite their strong performance, these Transformer-based models remain computationally intensive, limiting their applicability in mobile or real-time applications. Knowledge distillation has been employed to compress large SSL teachers into lightweight student models, yet most existing approaches retain Transformer architectures, thereby preserving the quadratic cost of attention mechanisms. In this work, we investigate MLP-Mixer–based student models as an alternative to Transformer students for SSL knowledge distillation. We propose a speech-oriented redesign of the MLP-Mixer that incorporates 1D convolution in the token-mixing stage to capture local temporal dependencies, and a Max-Feature-Map operation in the channel-mixing stage to emphasize speaker-discriminative frequency bands. Distilling knowledge from a WavLM teacher to this modified MLP-Mixer addresses the challenges of cross-architecture distillation. Extensive experiments on diverse benchmarks, including VoxCeleb, VCMix, VoxSRC, and VOiCES, demonstrate that the proposed approach achieves superior verification performance and computational efficiency compared to Transformer-based students, highlighting its potential for practical deployment.
- Open Access
- 25 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 111673-111682
Noise-Aware Extended U-Net With Split Encoder and Feature Refinement Module for Robust Speaker Verification in Noisy Environments
Speech data gathered from real-world environments typically contain noise, a significant element that undermines the performance of deep neural network-based speaker verification (SV) systems. To mitigate performance degradation due to noise and construct noise-robust SV systems, several researchers have employed frameworks that integrate speech enhancement (SE) and SV systems. We have proposed the extended U-Net (ExU-Net) that has achieved state-of-the-art performance in SV in noisy environments by jointly training SE and SV systems. In the SE field, some studies have shown that recognizing noise components within speech can improve the system’s performance. Inspired by these approaches, we propose a noise-aware ExU-Net (NA-ExU-Net) that acknowledges noise information in the SE process based on the ExU-Net architecture. The proposed system comprises a Split Encoder and a feature refinement module (FRM). The Split Encoder handles the speech and noise separately by dividing the encoder blocks, whereas FRM is designed to inhibit the propagation of irrelevant data via skip connections. To validate the effectiveness of our proposed framework in noisy conditions, we evaluated the models on the VoxCeleb1 test set with added noise from the MUSAN corpus. The experimental results demonstrate that NA-ExU-Net outperforms ExU-Net and other baseline systems under all evaluation conditions. Furthermore, evaluations in out-of-domain noise environments indicate that NA-ExU-Net significantly surpasses existing frameworks, highlighting its robustness and generalization capabilities. The codes utilized in our experiments can be accessed at https://github.com/chan-yeong0519/NA-ExU-Net.
- 25 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 111673-111682
Open AccessNoise-Aware Extended U-Net With Split Encoder and Feature Refinement Module for Robust Speaker Verification in Noisy Environments
Speech data gathered from real-world environments typically contain noise, a significant element that undermines the performance of deep neural network-based speaker verification (SV) systems. To mitigate performance degradation due to noise and construct noise-robust SV systems, several researchers have employed frameworks that integrate speech enhancement (SE) and SV systems. We have proposed the extended U-Net (ExU-Net) that has achieved state-of-the-art performance in SV in noisy environments by jointly training SE and SV systems. In the SE field, some studies have shown that recognizing noise components within speech can improve the system’s performance. Inspired by these approaches, we propose a noise-aware ExU-Net (NA-ExU-Net) that acknowledges noise information in the SE process based on the ExU-Net architecture. The proposed system comprises a Split Encoder and a feature refinement module (FRM). The Split Encoder handles the speech and noise separately by dividing the encoder blocks, whereas FRM is designed to inhibit the propagation of irrelevant data via skip connections. To validate the effectiveness of our proposed framework in noisy conditions, we evaluated the models on the VoxCeleb1 test set with added noise from the MUSAN corpus. The experimental results demonstrate that NA-ExU-Net outperforms ExU-Net and other baseline systems under all evaluation conditions. Furthermore, evaluations in out-of-domain noise environments indicate that NA-ExU-Net significantly surpasses existing frameworks, highlighting its robustness and generalization capabilities. The codes utilized in our experiments can be accessed at https://github.com/chan-yeong0519/NA-ExU-Net.
- 22 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 2269-2282
FA-ExU-Net: The Simultaneous Training of an Embedding Extractor and Enhancement Model for a Speaker Verification System Robust to Short Noisy Utterances
Speaker verification (SV) technology has the potential to enhance personalization and security in various applications, such as voice assistants, forensics, and access control. However, several challenges hinder the practical application of SV systems, including limitations and distortions in speaker information due to short utterances and noisy environments. Furthermore, these two factors often coexist in real-world situations, resulting in a significant performance degradation of SV systems. Despite the significance of these obstacles, each factor is independently studied, and the co-occurrence of both factors is rarely investigated. Here, we propose a novel SV framework, feature aggregated extended U-Net (FA-ExU-Net), which simultaneously addresses both the challenges by building on the success of prior research on each factor. The FA-ExU-Net incorporates an iterative and hierarchical feature aggregation scheme, a target task-specific feature enhancement module, and a multi-scale feature aggregator for extracting information-rich embeddings. Our proposed system outperforms the recent baseline models based on four evaluation criteria: generalizability, short utterance performance, capacity to handle noisy environments, and robustness to short utterances in noisy environments. We demonstrate the effectiveness of the proposed model through comparison and ablation experiments and intuitive visualizations. The proposed novel approach is expected to contribute to the development of more robust and accurate SV models for practical applications. Our training codes are available at https://github.com/wngh1187/FA-ExU-Net.
- 22 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 2269-2282
FA-ExU-Net: The Simultaneous Training of an Embedding Extractor and Enhancement Model for a Speaker Verification System Robust to Short Noisy Utterances
Speaker verification (SV) technology has the potential to enhance personalization and security in various applications, such as voice assistants, forensics, and access control. However, several challenges hinder the practical application of SV systems, including limitations and distortions in speaker information due to short utterances and noisy environments. Furthermore, these two factors often coexist in real-world situations, resulting in a significant performance degradation of SV systems. Despite the significance of these obstacles, each factor is independently studied, and the co-occurrence of both factors is rarely investigated. Here, we propose a novel SV framework, feature aggregated extended U-Net (FA-ExU-Net), which simultaneously addresses both the challenges by building on the success of prior research on each factor. The FA-ExU-Net incorporates an iterative and hierarchical feature aggregation scheme, a target task-specific feature enhancement module, and a multi-scale feature aggregator for extracting information-rich embeddings. Our proposed system outperforms the recent baseline models based on four evaluation criteria: generalizability, short utterance performance, capacity to handle noisy environments, and robustness to short utterances in noisy environments. We demonstrate the effectiveness of the proposed model through comparison and ablation experiments and intuitive visualizations. The proposed novel approach is expected to contribute to the development of more robust and accurate SV models for practical applications. Our training codes are available at https://github.com/wngh1187/FA-ExU-Net.