Hyunsoon Shin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Hyunsoon Shin has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- John Lorenzo Bautista
Emotion Recognition IoT Research Section, Hyper-Connected Communication Research Laboratory, Electronic and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
2 shared publications - Aziz Nasridinov
Dongguk University
1 shared publication - Boreom Lee
Gwangju Institute of Science and Technology
1 shared publication Chanki Park
Electronics and Telecommunications Research Institute, Yuseong-gu, Daejeon, South Korea
1 shared publication- Chanki Park
School of Mechanical Engineering, Gwangju Institute of Science and Technology (GIST), 123 Cheomdan-gwagiro, Buk-gu, Gwangju, 61005, South Korea
1 shared publication Hwayong Kim
School of Chemical Engineering and Institute of Chemical Processes, Seoul National University, San 56-1, Shilim-dong, Kwanak-ku,Seoul,Korea
1 shared publication
Latest Publications
- Open Access
- 10 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (2) , 623
Speech Emotion Recognition Model Based on Joint Modeling of Discrete and Dimensional Emotion Representation
This paper introduces a novel joint model architecture for Speech Emotion Recognition (SER) that integrates both discrete and dimensional emotional representations, allowing for the simultaneous training of classification and regression tasks to improve the comprehensiveness and interpretability of emotion recognition. By employing a joint loss function that combines categorical and regression losses, the model ensures balanced optimization across tasks, with experiments exploring various weighting schemes using a tunable parameter to adjust task importance. Two adaptive weight balancing schemes, Dynamic Weighting and Joint Weighting, further enhance performance by dynamically adjusting task weights based on optimization progress and ensuring balanced emotion representation during backpropagation. The architecture employs parallel feature extraction through independent encoders, designed to capture unique features from multiple modalities, including Mel-frequency Cepstral Coefficients (MFCC), Short-term Features (STF), Mel-spectrograms, and raw audio signals. Additionally, pre-trained models such as Wav2Vec 2.0 and HuBERT are integrated to leverage their robust latent features. The inclusion of self-attention and co-attention mechanisms allows the model to capture relationships between input modalities and interdependencies among features, further improving its interpretability and integration capabilities. Experiments conducted on the IEMOCAP dataset using a leave-one-subject-out approach demonstrate the model’s effectiveness, with results showing a 1–2% accuracy improvement over classification-only models. The optimal configuration, incorporating the joint architecture, dynamic weighting, and parallel processing of multimodal features, achieves a weighted accuracy of 72.66%, an unweighted accuracy of 73.22%, and a mean Concordance Correlation Coefficient (CCC) of 0.3717. These results validate the effectiveness of the proposed joint model architecture and adaptive balancing weight schemes in improving SER performance.
- 10 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (2) , 623
Open AccessSpeech Emotion Recognition Model Based on Joint Modeling of Discrete and Dimensional Emotion Representation
This paper introduces a novel joint model architecture for Speech Emotion Recognition (SER) that integrates both discrete and dimensional emotional representations, allowing for the simultaneous training of classification and regression tasks to improve the comprehensiveness and interpretability of emotion recognition. By employing a joint loss function that combines categorical and regression losses, the model ensures balanced optimization across tasks, with experiments exploring various weighting schemes using a tunable parameter to adjust task importance. Two adaptive weight balancing schemes, Dynamic Weighting and Joint Weighting, further enhance performance by dynamically adjusting task weights based on optimization progress and ensuring balanced emotion representation during backpropagation. The architecture employs parallel feature extraction through independent encoders, designed to capture unique features from multiple modalities, including Mel-frequency Cepstral Coefficients (MFCC), Short-term Features (STF), Mel-spectrograms, and raw audio signals. Additionally, pre-trained models such as Wav2Vec 2.0 and HuBERT are integrated to leverage their robust latent features. The inclusion of self-attention and co-attention mechanisms allows the model to capture relationships between input modalities and interdependencies among features, further improving its interpretability and integration capabilities. Experiments conducted on the IEMOCAP dataset using a leave-one-subject-out approach demonstrate the model’s effectiveness, with results showing a 1–2% accuracy improvement over classification-only models. The optimal configuration, incorporating the joint architecture, dynamic weighting, and parallel processing of multimodal features, achieves a weighted accuracy of 72.66%, an unweighted accuracy of 73.22%, and a mean Concordance Correlation Coefficient (CCC) of 0.3717. These results validate the effectiveness of the proposed joint model architecture and adaptive balancing weight schemes in improving SER performance.
- Open Access
- 24 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 41256-41263
High-Fidelity Compressive Heart Rate Tracking
Recently, photoplethysmography sensors in smart watches have been frequently used to monitor heart rate. Because a photoplethysmography sensor flashes light on the skin and measures the reflected light, the rate of flashing and sampling is directly related to its energy consumption. It is necessary to reduce the rate to extend the battery life of a smart watch. In this study, to satisfy sub-Nyquist sampling, real-time, and high accuracy requirements, we propose a novel heart rate tracking method that consists of online compressive covariance sensing, signal subspace tracking, and spectral peak tracking. The proposed method (average sampling rate: 1.4 Hz) showed better frequency tracking performance than the conventional spectral peak tracking algorithm (sampling rate: 10 Hz) in time-varying heart rate simulations, and a statistically significant difference between them was not observed in the real photoplethysmogram data. The trimmed average absolute error was 1.04 beats/min. As a result, the proposed real-time heart rate tracking method with sub-Nyquist sampling showed high accuracy.
- 24 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 41256-41263
Open AccessHigh-Fidelity Compressive Heart Rate Tracking
Recently, photoplethysmography sensors in smart watches have been frequently used to monitor heart rate. Because a photoplethysmography sensor flashes light on the skin and measures the reflected light, the rate of flashing and sampling is directly related to its energy consumption. It is necessary to reduce the rate to extend the battery life of a smart watch. In this study, to satisfy sub-Nyquist sampling, real-time, and high accuracy requirements, we propose a novel heart rate tracking method that consists of online compressive covariance sensing, signal subspace tracking, and spectral peak tracking. The proposed method (average sampling rate: 1.4 Hz) showed better frequency tracking performance than the conventional spectral peak tracking algorithm (sampling rate: 10 Hz) in time-varying heart rate simulations, and a statistically significant difference between them was not observed in the real photoplethysmogram data. The trimmed average absolute error was 1.04 beats/min. As a result, the proposed real-time heart rate tracking method with sub-Nyquist sampling showed high accuracy.
- Open Access
- 28 November 2022
- Published by MDPI AG in Electronics
- Vol. 11 (23) , 3935
Speech Emotion Recognition Based on Parallel CNN-Attention Networks with Multi-Fold Data Augmentation
In this paper, an automatic speech emotion recognition (SER) task of classifying eight different emotions was experimented using parallel based networks trained using the Ryeson Audio-Visual Dataset of Speech and Song (RAVDESS) dataset. A combination of a CNN-based network and attention-based networks, running in parallel, was used to model both spatial features and temporal feature representations. Multiple Augmentation techniques using Additive White Gaussian Noise (AWGN), SpecAugment, Room Impulse Response (RIR), and Tanh Distortion techniques were used to augment the training data to further generalize the model representation. Raw audio data were transformed into Mel-Spectrograms as the model’s input. Using CNN’s proven capability in image classification and spatial feature representations, the spectrograms were treated as an image with the height and width represented by the spectrogram’s time and frequency scales. Temporal feature representations were represented by attention-based models Transformer, and BLSTM-Attention modules. Proposed architectures of the parallel CNN-based networks running along with Transformer and BLSTM-Attention modules were compared with standalone CNN architectures and attention-based networks, as well as with hybrid architectures with CNN layers wrapped in time-distributed wrappers stacked on attention-based networks. In these experiments, the highest accuracy of 89.33% for a Parallel CNN-Transformer network and 85.67% for a Parallel CNN-BLSTM-Attention Network were achieved on a 10% hold-out test set from the dataset. These networks showed promising results based on their accuracies, while keeping significantly less training parameters compared with non-parallel hybrid models.
- 28 November 2022
- Published by MDPI AG in Electronics
- Vol. 11 (23) , 3935
Open AccessSpeech Emotion Recognition Based on Parallel CNN-Attention Networks with Multi-Fold Data Augmentation
In this paper, an automatic speech emotion recognition (SER) task of classifying eight different emotions was experimented using parallel based networks trained using the Ryeson Audio-Visual Dataset of Speech and Song (RAVDESS) dataset. A combination of a CNN-based network and attention-based networks, running in parallel, was used to model both spatial features and temporal feature representations. Multiple Augmentation techniques using Additive White Gaussian Noise (AWGN), SpecAugment, Room Impulse Response (RIR), and Tanh Distortion techniques were used to augment the training data to further generalize the model representation. Raw audio data were transformed into Mel-Spectrograms as the model’s input. Using CNN’s proven capability in image classification and spatial feature representations, the spectrograms were treated as an image with the height and width represented by the spectrogram’s time and frequency scales. Temporal feature representations were represented by attention-based models Transformer, and BLSTM-Attention modules. Proposed architectures of the parallel CNN-based networks running along with Transformer and BLSTM-Attention modules were compared with standalone CNN architectures and attention-based networks, as well as with hybrid architectures with CNN layers wrapped in time-distributed wrappers stacked on attention-based networks. In these experiments, the highest accuracy of 89.33% for a Parallel CNN-Transformer network and 85.67% for a Parallel CNN-BLSTM-Attention Network were achieved on a 10% hold-out test set from the dataset. These networks showed promising results based on their accuracies, while keeping significantly less training parameters compared with non-parallel hybrid models.
- Open Access
Blockwise PPG Enhancement Based on Time-Variant Zero-Phase Harmonic Notch Filtering
So far, many approaches have been developed for motion artifact (MA) reduction from photoplethysmogram (PPG). Specifically, single-input MA reduction methods are useful to apply wearable and mobile healthcare systems because of their low hardware costs and simplicity. However, most of them are insufficiently developed to be used in real-world situations, and they suffer from a phase distortion problem. In this study, we propose a novel single-input MA reduction algorithm based on time-variant forward-backward harmonic notch filtering. To verify the proposed method, we collected real PPG data corrupted by MA and compared it with existing single-input MA reduction methods. In conclusion, the proposed zero-phase line enhancer (ZLE) was found to be superior for MA reduction and exhibited zero phase response.
Open AccessBlockwise PPG Enhancement Based on Time-Variant Zero-Phase Harmonic Notch Filtering
So far, many approaches have been developed for motion artifact (MA) reduction from photoplethysmogram (PPG). Specifically, single-input MA reduction methods are useful to apply wearable and mobile healthcare systems because of their low hardware costs and simplicity. However, most of them are insufficiently developed to be used in real-world situations, and they suffer from a phase distortion problem. In this study, we propose a novel single-input MA reduction algorithm based on time-variant forward-backward harmonic notch filtering. To verify the proposed method, we collected real PPG data corrupted by MA and compared it with existing single-input MA reduction methods. In conclusion, the proposed zero-phase line enhancer (ZLE) was found to be superior for MA reduction and exhibited zero phase response.
- 1 January 2004
- Published by Royal Society of Chemistry (RSC) in Physical Chemistry Chemical Physics
- Vol. 6 (9) , 2295-2300
High pressure phase behavior of carbon dioxide + heptadecafluorodecyl acrylate and carbon dioxide + heptadecafluorodecyl methacrylate systems
We measured pressure–composition (P–x) isotherms for binary mixtures of carbon dioxide+ heptadecafluorodecyl acrylate and carbon dioxide+ heptadecafluorodecyl methacrylate using a variable volume view cell at temperatures from 323 K to 353 K and pressure up to 140 bar. Phase behavior of these binary experimental data was modeled with the Peng–Robinson equation of state with two adjustable binary interaction parameters. For the critical constants, we proposed new parameters for –CF2– and CF3– for the Joback group contribution method. These parameters were regressed from normal perfluoroalkane data. We compared this modified Joback group contribution method with the Constantinou–Gani group contribution method for correlation of experimental results.
- 1 January 2004
- Published by Royal Society of Chemistry (RSC) in Physical Chemistry Chemical Physics
- Vol. 6 (9) , 2295-2300
High pressure phase behavior of carbon dioxide + heptadecafluorodecyl acrylate and carbon dioxide + heptadecafluorodecyl methacrylate systems
We measured pressure–composition (P–x) isotherms for binary mixtures of carbon dioxide+ heptadecafluorodecyl acrylate and carbon dioxide+ heptadecafluorodecyl methacrylate using a variable volume view cell at temperatures from 323 K to 353 K and pressure up to 140 bar. Phase behavior of these binary experimental data was modeled with the Peng–Robinson equation of state with two adjustable binary interaction parameters. For the critical constants, we proposed new parameters for –CF2– and CF3– for the Joback group contribution method. These parameters were regressed from normal perfluoroalkane data. We compared this modified Joback group contribution method with the Constantinou–Gani group contribution method for correlation of experimental results.