Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Hyemi Kim has a total of 17 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Background music monitoring framework and dataset for TV broadcast audio

    by Hyemi Kim, Junghyun Kim, Jihyun Park, Seongwoo Kim, Chanjin Park, Wonyoung Yoo

    Music identification is widely regarded as a solved problem for music searching in quiet environments, but its performance tends to degrade in TV broadcast audio owing to the presence of dialogue or sound effects. In addition, constructing an accurate dataset for measuring the performance of background music monitoring in TV broadcast audio is challenging. We propose a framework for monitoring background music by automatic identification and introduce a background music cue sheet. The framework comprises three main components: music identification, music–speech separation, and music detection. In addition, we introduce the Cue‐K‐Drama dataset, which includes reference songs, audio tracks from 60 episodes of five Korean TV drama series, and corresponding cue sheets that provide the start and end timestamps of background music. Experimental results on the constructed and existing datasets demonstrate that the proposed framework, which incorporates music identification with music–speech separation and music detection, effectively enhances TV broadcast audio monitoring.


  • Open Access

    Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection

    by Sangwon Lee, Hyemi Kim, Gil-Jin Jang

    Sound event detection (SED) is the task of finding the identities of sound events, as well as their onset and offset timings from audio recordings. When complete timing information is not available in the training data, but only the event identities are known, SED should be solved by weakly supervised learning. The conventional U-Net with global weighted rank pooling (GWRP) has shown a decent performance, but extensive computation is demanded. We propose a novel U-Net with limited upsampling (LUU-Net) and global threshold average pooling (GTAP) to reduce the model size, as well as the computational overhead. The expansion along the frequency axis in the U-Net decoder was minimized, so that the output map sizes were reduced by 40% at the convolutional layers and 12.5% at the fully connected layers without SED performance degradation. The experimental results on a mixed dataset of DCASE 2018 Tasks 1 and 2 showed that our limited upsampling U-Net (LUU-Net) with GTAP was about 23% faster in training and achieved 0.644 in audio tagging and 0.531 in weakly supervised SED tasks in terms of F1 scores, while U-Net with GWRP showed 0.629 and 0.492, respectively. The major contribution of the proposed LUU-Net is the reduction in the computation time with the SED performance being maintained or improved. The other proposed method, GTAP, further improved the training time reduction and provides versatility for various audio mixing conditions by adjusting a single hyperparameter.


  • Open Access

    Integrating Dilated Convolution into DenseLSTM for Audio Source Separation

    by Woon-Haeng Heo, Hyemi Kim, Oh-Wook Kwon

    Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.


  • Open Access

    Source Separation Using Dilated Time-Frequency DenseNet for Music Identification in Broadcast Contents

    by Woon-Haeng Heo, Hyemi Kim, Oh-Wook Kwon

    We propose a source separation architecture using dilated time-frequency DenseNet for background music identification of broadcast content. We apply source separation techniques to the mixed signals of music and speech. For the source separation purpose, we propose a new architecture to add a time-frequency dilated convolution to the conventional DenseNet in order to effectively increase the receptive field in the source separation scheme. In addition, we apply different convolutions to each frequency band of the spectrogram in order to reflect the different frequency characteristics of the low- and high-frequency bands. To verify the performance of the proposed architecture, we perform singing-voice separation and music-identification experiments. As a result, we confirm that the proposed architecture produces the best performance in both experiments because it uses the dilated convolution to reflect wide contextual information.


  • ThunderPunch: A bare-hand, gesture-based, large interactive display interface with upper-body-part detection in a top view

    by Daehwan Kim, Hye-Mi Kim, Hang-Kee Kim, Su-Ran Park, Ki-Suk Lee, Ki-Hong Kim

    We present a new bare-hand gesture interface for large-screen interaction in which multiple users can participate simultaneously and interact with virtual content directly. To better reflect the intent of our new interface, we have created a new type of hardware system with a large hybrid display, named ThunderPunch. Unlike the conventional method, which involves positioning the camera in front, the cameras are mounted on the ceiling so that they avoid covering the large screen. To achieve bare-hand interaction in this hardware structure, we propose real-time algorithms that detect multiple body poses and recognize punching and touching gestures from top-view depth images. A pointing and touching test shows that the proposed algorithm is usable and that it outperforms other algorithms. In addition, we created a game to make the best use of the proposed system.


  • Face Recognition using 3D Line Edge Map Robust to Expression Changes and Occlusions

    by Hye-Mi Kim, Ho-Chul Shin, Seong Dae Kim

    We propose new face recognition algorithm using 3D line edge map as combining 3D information on the edge obtained by stereo camera and the conventional line edge map algorithm. As a similarity measure of 3D line edge map, weighted partial and spatially well-localized line segment Hausdorff distance (WPSLHD) is proposed. WPSLHD is a measure which is robust to expression changes, facial occlusions, and depth outliers. The proposed face recognition system performances are evaluated under expression changes and facial occlusions.