Sung-Woo Byun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Sung-Woo Byun has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
- Seok-Pil Lee
Department of Computer Science, Sangmyung University, Seoul 03016, Republic of Korea
6 shared publications - Ju-Hee Kim
Department of Computer Science, Graduate School, Sangmyung University, Seoul 03016, Korea
1 shared publication
Latest Publications
- Open Access
- 28 August 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (17) , 7967
Multi-Modal Emotion Recognition Using Speech Features and Text-Embedding
Recently, intelligent personal assistants, chat-bots and AI speakers are being utilized more broadly as communication interfaces and the demands for more natural interaction measures have increased as well. Humans can express emotions in various ways, such as using voice tones or facial expressions; therefore, multimodal approaches to recognize human emotions have been studied. In this paper, we propose an emotion recognition method to deliver more accuracy by using speech and text data. The strengths of the data are also utilized in this method. We conducted 43 feature vectors such as spectral features, harmonic features and MFCC from speech datasets. In addition, 256 embedding vectors from transcripts using pre-trained Tacotron encoder were extracted. The acoustic feature vectors and embedding vectors were fed into each deep learning model which produced a probability for the predicted output classes. The results show that the proposed model exhibited more accurate performance than in previous research.
- 28 August 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (17) , 7967
Open AccessMulti-Modal Emotion Recognition Using Speech Features and Text-Embedding
Recently, intelligent personal assistants, chat-bots and AI speakers are being utilized more broadly as communication interfaces and the demands for more natural interaction measures have increased as well. Humans can express emotions in various ways, such as using voice tones or facial expressions; therefore, multimodal approaches to recognize human emotions have been studied. In this paper, we propose an emotion recognition method to deliver more accuracy by using speech and text data. The strengths of the data are also utilized in this method. We conducted 43 feature vectors such as spectral features, harmonic features and MFCC from speech datasets. In addition, 256 embedding vectors from transcripts using pre-trained Tacotron encoder were extracted. The acoustic feature vectors and embedding vectors were fed into each deep learning model which produced a probability for the predicted output classes. The results show that the proposed model exhibited more accurate performance than in previous research.
- Open Access
- 21 February 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (4) , 1890
A Study on a Speech Emotion Recognition System with Effective Acoustic Features Using Deep Learning Algorithms
The goal of the human interface is to recognize the user’s emotional state precisely. In the speech emotion recognition study, the most important issue is the effective parallel use of the extraction of proper speech features and an appropriate classification engine. Well defined speech databases are also needed to accurately recognize and analyze emotions from speech signals. In this work, we constructed a Korean emotional speech database for speech emotion analysis and proposed a feature combination that can improve emotion recognition performance using a recurrent neural network model. To investigate the acoustic features, which can reflect distinct momentary changes in emotional expression, we extracted F0, Mel-frequency cepstrum coefficients, spectral features, harmonic features, and others. Statistical analysis was performed to select an optimal combination of acoustic features that affect the emotion from speech. We used a recurrent neural network model to classify emotions from speech. The results show the proposed system has more accurate performance than previous studies.
- 21 February 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (4) , 1890
Open AccessA Study on a Speech Emotion Recognition System with Effective Acoustic Features Using Deep Learning Algorithms
The goal of the human interface is to recognize the user’s emotional state precisely. In the speech emotion recognition study, the most important issue is the effective parallel use of the extraction of proper speech features and an appropriate classification engine. Well defined speech databases are also needed to accurately recognize and analyze emotions from speech signals. In this work, we constructed a Korean emotional speech database for speech emotion analysis and proposed a feature combination that can improve emotion recognition performance using a recurrent neural network model. To investigate the acoustic features, which can reflect distinct momentary changes in emotional expression, we extracted F0, Mel-frequency cepstrum coefficients, spectral features, harmonic features, and others. Statistical analysis was performed to select an optimal combination of acoustic features that affect the emotion from speech. We used a recurrent neural network model to classify emotions from speech. The results show the proposed system has more accurate performance than previous studies.
- Open Access
- 26 January 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (3) , 1144
Design of a Multi-Condition Emotional Speech Synthesizer
Recently, researchers have developed text-to-speech models based on deep learning, which have produced results superior to those of previous approaches. However, because those systems only mimic the generic speaking style of reference audio, it is difficult to assign user-defined emotional types to synthesized speech. This paper proposes an emotional speech synthesizer constructed by embedding not only speaking styles but also emotional styles. We extend speaker embedding to multi-condition embedding by adding emotional embedding in Tacotron, so that the synthesizer can generate emotional speech. An evaluation of the results showed the superiority of the proposed model to a previous model, in terms of emotional expressiveness.
- 26 January 2021
- Published by MDPI AG in Applied Sciences
- Vol. 11 (3) , 1144
Open AccessDesign of a Multi-Condition Emotional Speech Synthesizer
Recently, researchers have developed text-to-speech models based on deep learning, which have produced results superior to those of previous approaches. However, because those systems only mimic the generic speaking style of reference audio, it is difficult to assign user-defined emotional types to synthesized speech. This paper proposes an emotional speech synthesizer constructed by embedding not only speaking styles but also emotional styles. We extend speaker embedding to multi-condition embedding by adding emotional embedding in Tacotron, so that the synthesizer can generate emotional speech. An evaluation of the results showed the superiority of the proposed model to a previous model, in terms of emotional expressiveness.
- Open Access
- 18 September 2020
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 80 (28-29) , 35871-35885
Human emotion recognition based on the weighted integration method using image sequences and acoustic features
People generally perceive other people’s emotions based on speech and facial expressions, so it can be helpful to use speech signals and facial images simultaneously. However, because the characteristics of speech and image data are different, combining the two inputs is still a challenging issue in the area of emotion-recognition research. In this paper, we propose a method to recognize emotions by synchronizing speech signals and image sequences. We design three deep networks. One of the networks is trained using image sequences, which focus on facial expression changes. Facial landmarks are also input to another network to reflect facial motion. The speech signals are first converted to acoustic features, which are used for the input of the other network, synchronizing the image sequence. These three networks are combined using a novel integration method to boost the performance of emotion recognition. A test comparing accuracy is conducted to verify the proposed method. The results demonstrated that the proposed method exhibits more accurate performance than previous studies.
- 18 September 2020
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 80 (28-29) , 35871-35885
Open AccessHuman emotion recognition based on the weighted integration method using image sequences and acoustic features
People generally perceive other people’s emotions based on speech and facial expressions, so it can be helpful to use speech signals and facial images simultaneously. However, because the characteristics of speech and image data are different, combining the two inputs is still a challenging issue in the area of emotion-recognition research. In this paper, we propose a method to recognize emotions by synchronizing speech signals and image sequences. We design three deep networks. One of the networks is trained using image sequences, which focus on facial expression changes. Facial landmarks are also input to another network to reflect facial motion. The speech signals are first converted to acoustic features, which are used for the input of the other network, synchronizing the image sequence. These three networks are combined using a novel integration method to boost the performance of emotion recognition. A test comparing accuracy is conducted to verify the proposed method. The results demonstrated that the proposed method exhibits more accurate performance than previous studies.
- Open Access
- 12 October 2019
- Published by MDPI AG in Micromachines
- Vol. 10 (10) , 692
Implementation of Hand Gesture Recognition Device Applicable to Smart Watch Based on Flexible Epidermal Tactile Sensor Array
Ever since the development of digital devices, the recognition of human gestures has played an important role in many Human-Computer interface applications. Various wearable devices have been developed, and inertial sensors, magnetic sensors, gyro sensors, electromyography, force-sensitive resistors, and other types of sensors have been used to identify gestures. However, there are different drawbacks for each sensor, which affect the detection of gestures. In this paper, we present a new gesture recognition method using a Flexible Epidermal Tactile Sensor based on strain gauges to sense deformation. Such deformations are transduced to electric signals. By measuring the electric signals, the sensor can estimate the degree of deformation, including compression, tension, and twist, caused by movements of the wrist. The proposed sensor array was demonstrated to be capable of analyzing the eight motions of the wrist, and showed robustness, stability, and repeatability throughout a range of experiments aimed at testing the sensor array. We compared the performance of the prototype device with those of previous studies, under the same experimental conditions. The result shows our recognition method significantly outperformed existing methods.
- 12 October 2019
- Published by MDPI AG in Micromachines
- Vol. 10 (10) , 692
Open AccessImplementation of Hand Gesture Recognition Device Applicable to Smart Watch Based on Flexible Epidermal Tactile Sensor Array
Ever since the development of digital devices, the recognition of human gestures has played an important role in many Human-Computer interface applications. Various wearable devices have been developed, and inertial sensors, magnetic sensors, gyro sensors, electromyography, force-sensitive resistors, and other types of sensors have been used to identify gestures. However, there are different drawbacks for each sensor, which affect the detection of gestures. In this paper, we present a new gesture recognition method using a Flexible Epidermal Tactile Sensor based on strain gauges to sense deformation. Such deformations are transduced to electric signals. By measuring the electric signals, the sensor can estimate the degree of deformation, including compression, tension, and twist, caused by movements of the wrist. The proposed sensor array was demonstrated to be capable of analyzing the eight motions of the wrist, and showed robustness, stability, and repeatability throughout a range of experiments aimed at testing the sensor array. We compared the performance of the prototype device with those of previous studies, under the same experimental conditions. The result shows our recognition method significantly outperformed existing methods.
- 31 August 2019
- Published by Korean Society for Internet Information (KSII) in KSII Transactions on Internet and Information Systems
- Vol. 13 (8) , 4300-4314
Stochastic Non-linear Hashing for Near-Duplicate Video Retrieval using Deep Feature applicable to Large-scale Datasets
- 31 August 2019
- Published by Korean Society for Internet Information (KSII) in KSII Transactions on Internet and Information Systems
- Vol. 13 (8) , 4300-4314
Stochastic Non-linear Hashing for Near-Duplicate Video Retrieval using Deep Feature applicable to Large-scale Datasets