Hyemi Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hyemi Kim has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
Jihyun Park
Content Research Division Electronics and Telecommunications Research Institute Daejeon Republic of Korea
2 shared publicationsJunghyun Kim
Content Research Division Electronics and Telecommunications Research Institute Daejeon Republic of Korea
2 shared publications- Oh-Wook Kwon
School of Electronics Engineering, Chungbuk National University
2 shared publications Seongwoo Kim
Department of Electronic Engineering Inha University Incheon Republic of Korea
2 shared publicationsWonyoung Yoo
Content Research Division Electronics and Telecommunications Research Institute Daejeon Republic of Korea
2 shared publications- Woon-Haeng Heo
School of Electronics Engineering, Chungbuk National University, Cheongju 28644, Korea
2 shared publications
Latest Publications
- 12 April 2024
- Published by Wiley in ETRI Journal
- Vol. 46 (4) , 697-707
Background music monitoring framework and dataset for TV broadcast audio
Music identification is widely regarded as a solved problem for music searching in quiet environments, but its performance tends to degrade in TV broadcast audio owing to the presence of dialogue or sound effects. In addition, constructing an accurate dataset for measuring the performance of background music monitoring in TV broadcast audio is challenging. We propose a framework for monitoring background music by automatic identification and introduce a background music cue sheet. The framework comprises three main components: music identification, music–speech separation, and music detection. In addition, we introduce the Cue‐K‐Drama dataset, which includes reference songs, audio tracks from 60 episodes of five Korean TV drama series, and corresponding cue sheets that provide the start and end timestamps of background music. Experimental results on the constructed and existing datasets demonstrate that the proposed framework, which incorporates music identification with music–speech separation and music detection, effectively enhances TV broadcast audio monitoring.
- 12 April 2024
- Published by Wiley in ETRI Journal
- Vol. 46 (4) , 697-707
Background music monitoring framework and dataset for TV broadcast audio
Music identification is widely regarded as a solved problem for music searching in quiet environments, but its performance tends to degrade in TV broadcast audio owing to the presence of dialogue or sound effects. In addition, constructing an accurate dataset for measuring the performance of background music monitoring in TV broadcast audio is challenging. We propose a framework for monitoring background music by automatic identification and introduce a background music cue sheet. The framework comprises three main components: music identification, music–speech separation, and music detection. In addition, we introduce the Cue‐K‐Drama dataset, which includes reference songs, audio tracks from 60 episodes of five Korean TV drama series, and corresponding cue sheets that provide the start and end timestamps of background music. Experimental results on the constructed and existing datasets demonstrate that the proposed framework, which incorporates music identification with music–speech separation and music detection, effectively enhances TV broadcast audio monitoring.
- Open Access
- 4 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (11) , 6822
Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection
Sound event detection (SED) is the task of finding the identities of sound events, as well as their onset and offset timings from audio recordings. When complete timing information is not available in the training data, but only the event identities are known, SED should be solved by weakly supervised learning. The conventional U-Net with global weighted rank pooling (GWRP) has shown a decent performance, but extensive computation is demanded. We propose a novel U-Net with limited upsampling (LUU-Net) and global threshold average pooling (GTAP) to reduce the model size, as well as the computational overhead. The expansion along the frequency axis in the U-Net decoder was minimized, so that the output map sizes were reduced by 40% at the convolutional layers and 12.5% at the fully connected layers without SED performance degradation. The experimental results on a mixed dataset of DCASE 2018 Tasks 1 and 2 showed that our limited upsampling U-Net (LUU-Net) with GTAP was about 23% faster in training and achieved 0.644 in audio tagging and 0.531 in weakly supervised SED tasks in terms of F1 scores, while U-Net with GWRP showed 0.629 and 0.492, respectively. The major contribution of the proposed LUU-Net is the reduction in the computation time with the SED performance being maintained or improved. The other proposed method, GTAP, further improved the training time reduction and provides versatility for various audio mixing conditions by adjusting a single hyperparameter.
- 4 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (11) , 6822
Open AccessWeakly Supervised U-Net with Limited Upsampling for Sound Event Detection
Sound event detection (SED) is the task of finding the identities of sound events, as well as their onset and offset timings from audio recordings. When complete timing information is not available in the training data, but only the event identities are known, SED should be solved by weakly supervised learning. The conventional U-Net with global weighted rank pooling (GWRP) has shown a decent performance, but extensive computation is demanded. We propose a novel U-Net with limited upsampling (LUU-Net) and global threshold average pooling (GTAP) to reduce the model size, as well as the computational overhead. The expansion along the frequency axis in the U-Net decoder was minimized, so that the output map sizes were reduced by 40% at the convolutional layers and 12.5% at the fully connected layers without SED performance degradation. The experimental results on a mixed dataset of DCASE 2018 Tasks 1 and 2 showed that our limited upsampling U-Net (LUU-Net) with GTAP was about 23% faster in training and achieved 0.644 in audio tagging and 0.531 in weakly supervised SED tasks in terms of F1 scores, while U-Net with GWRP showed 0.629 and 0.492, respectively. The major contribution of the proposed LUU-Net is the reduction in the computation time with the SED performance being maintained or improved. The other proposed method, GTAP, further improved the training time reduction and provides versatility for various audio mixing conditions by adjusting a single hyperparameter.
- Open Access
- 31 January 2023
- Published by Digital Contents Society in Journal of Digital Contents Society
- Vol. 24 (1) , 59-68
Pseudo-Broadcast Music-Speech and Cuesheet Dataset for Background Music Identification/Separation/Detection in TV Broadcast Audio
- 31 January 2023
- Published by Digital Contents Society in Journal of Digital Contents Society
- Vol. 24 (1) , 59-68
Open AccessPseudo-Broadcast Music-Speech and Cuesheet Dataset for Background Music Identification/Separation/Detection in TV Broadcast Audio
- Open Access
- 15 January 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (2) , 789
Integrating Dilated Convolution into DenseLSTM for Audio Source Separation
Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.
- 15 January 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (2) , 789
Open AccessIntegrating Dilated Convolution into DenseLSTM for Audio Source Separation
Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.
- Open Access
- 3 March 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (5) , 1727
Source Separation Using Dilated Time-Frequency DenseNet for Music Identification in Broadcast Contents
We propose a source separation architecture using dilated time-frequency DenseNet for background music identification of broadcast content. We apply source separation techniques to the mixed signals of music and speech. For the source separation purpose, we propose a new architecture to add a time-frequency dilated convolution to the conventional DenseNet in order to effectively increase the receptive field in the source separation scheme. In addition, we apply different convolutions to each frequency band of the spectrogram in order to reflect the different frequency characteristics of the low- and high-frequency bands. To verify the performance of the proposed architecture, we perform singing-voice separation and music-identification experiments. As a result, we confirm that the proposed architecture produces the best performance in both experiments because it uses the dilated convolution to reflect wide contextual information.
- 3 March 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (5) , 1727
Open AccessSource Separation Using Dilated Time-Frequency DenseNet for Music Identification in Broadcast Contents
We propose a source separation architecture using dilated time-frequency DenseNet for background music identification of broadcast content. We apply source separation techniques to the mixed signals of music and speech. For the source separation purpose, we propose a new architecture to add a time-frequency dilated convolution to the conventional DenseNet in order to effectively increase the receptive field in the source separation scheme. In addition, we apply different convolutions to each frequency band of the spectrogram in order to reflect the different frequency characteristics of the low- and high-frequency bands. To verify the performance of the proposed architecture, we perform singing-voice separation and music-identification experiments. As a result, we confirm that the proposed architecture produces the best performance in both experiments because it uses the dilated convolution to reflect wide contextual information.
- 27 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 38 (5) , 100-111
ThunderPunch: A bare-hand, gesture-based, large interactive display interface with upper-body-part detection in a top view
We present a new bare-hand gesture interface for large-screen interaction in which multiple users can participate simultaneously and interact with virtual content directly. To better reflect the intent of our new interface, we have created a new type of hardware system with a large hybrid display, named ThunderPunch. Unlike the conventional method, which involves positioning the camera in front, the cameras are mounted on the ceiling so that they avoid covering the large screen. To achieve bare-hand interaction in this hardware structure, we propose real-time algorithms that detect multiple body poses and recognize punching and touching gestures from top-view depth images. A pointing and touching test shows that the proposed algorithm is usable and that it outperforms other algorithms. In addition, we created a game to make the best use of the proposed system.
- 27 September 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 38 (5) , 100-111
ThunderPunch: A bare-hand, gesture-based, large interactive display interface with upper-body-part detection in a top view
We present a new bare-hand gesture interface for large-screen interaction in which multiple users can participate simultaneously and interact with virtual content directly. To better reflect the intent of our new interface, we have created a new type of hardware system with a large hybrid display, named ThunderPunch. Unlike the conventional method, which involves positioning the camera in front, the cameras are mounted on the ceiling so that they avoid covering the large screen. To achieve bare-hand interaction in this hardware structure, we propose real-time algorithms that detect multiple body poses and recognize punching and touching gestures from top-view depth images. A pointing and touching test shows that the proposed algorithm is usable and that it outperforms other algorithms. In addition, we created a game to make the best use of the proposed system.
- 1 February 2007
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (17389445) , 357-362
Face Recognition using 3D Line Edge Map Robust to Expression Changes and Occlusions
We propose new face recognition algorithm using 3D line edge map as combining 3D information on the edge obtained by stereo camera and the conventional line edge map algorithm. As a similarity measure of 3D line edge map, weighted partial and spatially well-localized line segment Hausdorff distance (WPSLHD) is proposed. WPSLHD is a measure which is robust to expression changes, facial occlusions, and depth outliers. The proposed face recognition system performances are evaluated under expression changes and facial occlusions.
- 1 February 2007
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (17389445) , 357-362
Face Recognition using 3D Line Edge Map Robust to Expression Changes and Occlusions
We propose new face recognition algorithm using 3D line edge map as combining 3D information on the edge obtained by stereo camera and the conventional line edge map algorithm. As a similarity measure of 3D line edge map, weighted partial and spatially well-localized line segment Hausdorff distance (WPSLHD) is proposed. WPSLHD is a measure which is robust to expression changes, facial occlusions, and depth outliers. The proposed face recognition system performances are evaluated under expression changes and facial occlusions.