Daewoong Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Daewoong Kim has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
- Danbinaerin Han
Graduate School of Culture Technology, KAIST, Daejeon, the Republic of Korea
1 shared publication - Dasaem Jeong
Sogang University, Seoul, South Korea
1 shared publication
Latest Publications
- Open Access
- 10 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 7-11
Aligning Incomplete Lyrics of Korean Folk Song Dataset using Whisper
In this study, we introduce a method for time-alignment of lyrics in Korean folk song audio using a transformer encoder-decoder model specifically designed to utilize incomplete lyric data. We analyzed the characteristics of Korean folk song lyrics and found some discrepancies between the lyrics and the corresponding audio recordings. To address these challenges and maximize the use of existing transcriptions, we introduce RefWhisper. This is a variant of OpenAI’s Whisper and includes an extra encoder module and cross-attention layer, enabling the model to consult incomplete lyrics during the transcription process. The added cross-attention layer facilitates not only the alignment of the reference text with the predicted transcription but also with the audio. We make public the transcribed outcomes and timestamp data, which are aligned at both the sentence and word levels, for a corpus of 13,801 Korean folk songs.
- 10 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 7-11
Open AccessAligning Incomplete Lyrics of Korean Folk Song Dataset using Whisper
In this study, we introduce a method for time-alignment of lyrics in Korean folk song audio using a transformer encoder-decoder model specifically designed to utilize incomplete lyric data. We analyzed the characteristics of Korean folk song lyrics and found some discrepancies between the lyrics and the corresponding audio recordings. To address these challenges and maximize the use of existing transcriptions, we introduce RefWhisper. This is a variant of OpenAI’s Whisper and includes an extra encoder module and cross-attention layer, enabling the model to consult incomplete lyrics during the transcription process. The added cross-attention layer facilitates not only the alignment of the reference text with the predicted transcription but also with the audio. We make public the transcribed outcomes and timestamp data, which are aligned at both the sentence and word levels, for a corpus of 13,801 Korean folk songs.