Youshin Lim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Youshin Lim has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Byeong-Yeol Kim
42dot Inc., Hyundai Motor Company, Seoul, Republic of Korea
3 shared publications - Joon Son Chung
KAIST, Daejeon, Republic of Korea
2 shared publications - Hanbin Lee
Department of Statistics, University of Michigan, Ann Arbor, MI, 48109, USA
1 shared publication - Hyeongkeun Lee
KAIST, Daejeon, Republic of Korea
1 shared publication - Hyung Yong Kim
Seoul National University, Seoul, South Korea
1 shared publication - Jaemin Jung
Moon Soul Graduate School of Future Strategy, Korea Advanced Institute of Science and Technology, Republic of Korea
1 shared publication
Latest Publications
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3827-3836
That's What I Said: Fully-Controllable Talking Face Generation
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3827-3836
Open AccessThat's What I Said: Fully-Controllable Talking Face Generation
- 9 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9-14
ASBERT: ASR-Specific Self-Supervised Learning with Self-Training
Pre-training of self-supervised learning (SSL) generally shows a good performance on various speech processing tasks. However, this pre-training scheme may lead to a sub-optimal solution for fine-tuning a specific task, such as automatic speech recognition (ASR). In order to provide a more optimal pre-trained model for ASR, we introduce an ASR-Specific hidden-unit BERT with self-training, namely ASBERT. Motivated by self-training, we extract linguistic-related pseudo labels from the fine-tuned model, and these labels are used in the next pre-training procedure. Experimental results on LibriSpeech test-clean and test-other datasets show that ASBERT without language model (LM) outperforms the conventional SSL and self-training model, achieving a 6.3/2.0% and 15.4/13.2% relatively word error rate reduction (RERR). Moreover, without using pseudo-transcription, ASBERT yields comparable performance to the conventional self-training method.
- 9 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9-14
ASBERT: ASR-Specific Self-Supervised Learning with Self-Training
Pre-training of self-supervised learning (SSL) generally shows a good performance on various speech processing tasks. However, this pre-training scheme may lead to a sub-optimal solution for fine-tuning a specific task, such as automatic speech recognition (ASR). In order to provide a more optimal pre-trained model for ASR, we introduce an ASR-Specific hidden-unit BERT with self-training, namely ASBERT. Motivated by self-training, we extract linguistic-related pseudo labels from the fine-tuned model, and these labels are used in the next pre-training procedure. Experimental results on LibriSpeech test-clean and test-other datasets show that ASBERT without language model (LM) outperforms the conventional SSL and self-training model, achieving a 6.3/2.0% and 15.4/13.2% relatively word error rate reduction (RERR). Moreover, without using pseudo-transcription, ASBERT yields comparable performance to the conventional self-training method.
- Open Access
Metric Learning for User-defined Keyword Spotting
The goal of this work is to detect new spoken terms defined by users. While most previous works address Keyword Spotting (KWS) as a closed-set classification problem, this limits their transferability to unseen terms. The ability to define custom keywords has advantages in terms of user experience. In this paper, we propose a metric learning-based training strategy for user-defined keyword spotting. In particular, we make the following contributions: (1) we construct a large-scale keyword dataset with an existing speech corpus and propose a filtering method to remove data that degrade model training; (2) we propose a metric learning-based two-stage training strategy, and demonstrate that the proposed method improves the performance on the user-defined keyword spotting task by enriching their representations; (3) to facilitate the fair comparison in the user-defined KWS field, we propose unified evaluation protocol and metrics. Our proposed system does not require an incremental training on the user-defined keywords, and outperforms previous works by a significant margin on the Google Speech Commands dataset using the proposed as well as the existing metrics.
Open AccessMetric Learning for User-defined Keyword Spotting
The goal of this work is to detect new spoken terms defined by users. While most previous works address Keyword Spotting (KWS) as a closed-set classification problem, this limits their transferability to unseen terms. The ability to define custom keywords has advantages in terms of user experience. In this paper, we propose a metric learning-based training strategy for user-defined keyword spotting. In particular, we make the following contributions: (1) we construct a large-scale keyword dataset with an existing speech corpus and propose a filtering method to remove data that degrade model training; (2) we propose a metric learning-based two-stage training strategy, and demonstrate that the proposed method improves the performance on the user-defined keyword spotting task by enriching their representations; (3) to facilitate the fair comparison in the user-defined KWS field, we propose unified evaluation protocol and metrics. Our proposed system does not require an incremental training on the user-defined keywords, and outperforms previous works by a significant margin on the Google Speech Commands dataset using the proposed as well as the existing metrics.