Hwa Jeon Song
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Hwa Jeon Song has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Byunghyun Yoo
Electronics and Telecommunications Research Institute (ETRI) Daejeon South Korea
2 shared publications Hyun Woo Kim
Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea
2 shared publicationsByounghwa Lee
Electronics and Telecommunications Research Institute (ETRI)
1 shared publication- Chun-Hee Lee
School of Computer Science and Engineering, Kyungpook National University, Daegu, Republic of Korea
1 shared publication Devarani Devi Ningombam
Department of Computer Science and Engineering, GITAM University, Visakhapatnam, India
1 shared publication- Devarani Devi Ningombam
Department of Computer Engineering, Chosun University, 309 Pilmun-daero, Dong-gu, Gwangju 61452, Korea
1 shared publication
Latest Publications
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- p. 1
Internal Temporal Conditioning of Selective State-Space Models for Event-Time Prediction: A Controlled Evaluation
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- p. 1
Open AccessInternal Temporal Conditioning of Selective State-Space Models for Event-Time Prediction: A Controlled Evaluation
- 28 March 2024
- Published by Elsevier in Journal of Parallel and Distributed Computing
- Vol. 190
Fast knowledge graph completion using graphics processing units
- 28 March 2024
- Published by Elsevier in Journal of Parallel and Distributed Computing
- Vol. 190
Fast knowledge graph completion using graphics processing units
- Open Access
- 18 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 116561-116569
Modeling Long-Term Multimodal Representations for Active Speaker Detection With Spatio-Positional Encoder
In this study, we present an end-to-end framework for active speaker detection to achieve robust performance in challenging scenarios with multiple speakers. In contrast to recent approaches, which rely heavily on the visual relational context between all speakers in a video frame, we propose collaboratively learning multimodal representations based on the audio and visual signals of a single candidate. Firstly, we propose a spatio-positional encoder to effectively address the problem of false detections caused by indiscernible faces in a video frame. Secondly, we present an efficient multimodal approach that models the long-term temporal contextual interactions between audio and visual modalities. Through extensive experiments on the AVA-ActiveSpeaker dataset, we demonstrate that our framework notably outperforms recent state-of-the-art approaches under challenging multi-speaker settings. Additionally, the proposed framework significantly improves the robustness against auditory and visual noise interference without relying on pre-trained networks or hand-crafted training strategies.
- 18 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 116561-116569
Open AccessModeling Long-Term Multimodal Representations for Active Speaker Detection With Spatio-Positional Encoder
In this study, we present an end-to-end framework for active speaker detection to achieve robust performance in challenging scenarios with multiple speakers. In contrast to recent approaches, which rely heavily on the visual relational context between all speakers in a video frame, we propose collaboratively learning multimodal representations based on the audio and visual signals of a single candidate. Firstly, we propose a spatio-positional encoder to effectively address the problem of false detections caused by indiscernible faces in a video frame. Secondly, we present an efficient multimodal approach that models the long-term temporal contextual interactions between audio and visual modalities. Through extensive experiments on the AVA-ActiveSpeaker dataset, we demonstrate that our framework notably outperforms recent state-of-the-art approaches under challenging multi-speaker settings. Additionally, the proposed framework significantly improves the robustness against auditory and visual noise interference without relying on pre-trained networks or hand-crafted training strategies.
- Open Access
- 16 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 87254-87265
CuMARL: Curiosity-Based Learning in Multiagent Reinforcement Learning
In this paper, we propose a novel curiosity-based learning algorithm for Multi-agent Reinforcement Learning (MARL) to attain efficient and effective decision-making. We employ the centralized training with decentralized execution framework (CTDE) and consider that each agent has knowledge of the prior action distribution of others. To quantify the difference in agents’ knowledge, curiosity, we introduce conditional mutual information (CMI) regularization and use the amount of information for updating decision-making policy. Then, to deploy these learning frameworks in a large-scale MARL setting while acquiring high sample efficiency, we consider a Kullback-Leibler (KL) divergence-based prioritization of experiences. We evaluate the effectiveness of the proposed algorithm in three different levels of StarCraft Multi Agent Challenge (SMAC) scenarios using the PyMARL framework. The simulation-based performance analysis shows that the proposed technique significantly improves the test win rate compared to various state-of-the-art MARL benchmarks, such as the Optimistically Weighted Monotonic Value Function Factorization (OW_QMIX) and Learning Individual Intrinsic Reward (LIIR).
- 16 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 87254-87265
Open AccessCuMARL: Curiosity-Based Learning in Multiagent Reinforcement Learning
In this paper, we propose a novel curiosity-based learning algorithm for Multi-agent Reinforcement Learning (MARL) to attain efficient and effective decision-making. We employ the centralized training with decentralized execution framework (CTDE) and consider that each agent has knowledge of the prior action distribution of others. To quantify the difference in agents’ knowledge, curiosity, we introduce conditional mutual information (CMI) regularization and use the amount of information for updating decision-making policy. Then, to deploy these learning frameworks in a large-scale MARL setting while acquiring high sample efficiency, we consider a Kullback-Leibler (KL) divergence-based prioritization of experiences. We evaluate the effectiveness of the proposed algorithm in three different levels of StarCraft Multi Agent Challenge (SMAC) scenarios using the PyMARL framework. The simulation-based performance analysis shows that the proposed technique significantly improves the test win rate compared to various state-of-the-art MARL benchmarks, such as the Optimistically Weighted Monotonic Value Function Factorization (OW_QMIX) and Learning Individual Intrinsic Reward (LIIR).
- Open Access
- 28 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 47741-47753
A Novel and Efficient Influence-Seeking Exploration in Deep Multiagent Reinforcement Learning
Although recent years witnessed notable success for a cooperative setting in multi-agent reinforcement learning (MARL), efficient explorations are still challenging primarily due to the complex dynamics of inter-agent interactions constituting the high dimension of action spaces. For an efficient exploration, it is necessary to quantify influences that can represent interactions among agents and use them to obtain more information about the complexity of multi-agent systems. In this paper, we propose a novel influence-seeking exploration (ISE) scheme, which encourages agents to preferably explore action spaces significantly influenced by others and thus helps in speeding up the learning curve. To measure the influence of other agents in action selection, we use the variance of joint action-values with different action sets of agents that obtained by an estimation technique to lessen computation overhead. To this end, we first present an analytical approach inspired by the concept of approximated variance propagation and then apply it to an exploration scheme. We evaluate the proposed exploration method on a set of StarCraft II micromanagement as well as modified predator-prey tasks. Compared to state-of-the-art methods, the proposed method achieved performance improvements of 10% in StarCraft II micromanagement and 50% in modified predator-prey tasks approximately.
- 28 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 47741-47753
Open AccessA Novel and Efficient Influence-Seeking Exploration in Deep Multiagent Reinforcement Learning
Although recent years witnessed notable success for a cooperative setting in multi-agent reinforcement learning (MARL), efficient explorations are still challenging primarily due to the complex dynamics of inter-agent interactions constituting the high dimension of action spaces. For an efficient exploration, it is necessary to quantify influences that can represent interactions among agents and use them to obtain more information about the complexity of multi-agent systems. In this paper, we propose a novel influence-seeking exploration (ISE) scheme, which encourages agents to preferably explore action spaces significantly influenced by others and thus helps in speeding up the learning curve. To measure the influence of other agents in action selection, we use the variance of joint action-values with different action sets of agents that obtained by an estimation technique to lessen computation overhead. To this end, we first present an analytical approach inspired by the concept of approximated variance propagation and then apply it to an exploration scheme. We evaluate the proposed exploration method on a set of StarCraft II micromanagement as well as modified predator-prey tasks. Compared to state-of-the-art methods, the proposed method achieved performance improvements of 10% in StarCraft II micromanagement and 50% in modified predator-prey tasks approximately.
- Open Access
- 22 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 34236-34245
Imbalanced Classification via Feature Dictionary-Based Minority Oversampling
Image classification research is one of the fields continuously studied in the computer vision domain, and several related studies have been actively conducted until recently. However, a limit exists regarding the prediction performance of real-world datasets due to the data imbalance problem between classes. Data augmentation through artificial sample generation for minority classes is one of the methods used to overcome this limitation. Among the various oversampling methods, we propose the feature dictionary-based generative model for the oversampling method. Feature dictionaries are built through the pretrained feature extractor, and the proposed generative model synthesizes artificial samples based on the dictionary. Class-to-class balanced training can be conducted by fine-tuning the classifier as additional data for the minority class. We experiment by applying the proposed framework to the fashion dataset, which has an extreme class imbalance. The experimental results demonstrate that the proposed model achieved the highest top-1 performance on various public fashion datasets. In addition, we analyze the number of samples in the dictionary and test the effectiveness of the elements that comprise the proposed model using various ablation studies.
- 22 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 34236-34245
Open AccessImbalanced Classification via Feature Dictionary-Based Minority Oversampling
Image classification research is one of the fields continuously studied in the computer vision domain, and several related studies have been actively conducted until recently. However, a limit exists regarding the prediction performance of real-world datasets due to the data imbalance problem between classes. Data augmentation through artificial sample generation for minority classes is one of the methods used to overcome this limitation. Among the various oversampling methods, we propose the feature dictionary-based generative model for the oversampling method. Feature dictionaries are built through the pretrained feature extractor, and the proposed generative model synthesizes artificial samples based on the dictionary. Class-to-class balanced training can be conducted by fine-tuning the classifier as additional data for the minority class. We experiment by applying the proposed framework to the fashion dataset, which has an extreme class imbalance. The experimental results demonstrate that the proposed model achieved the highest top-1 performance on various public fashion datasets. In addition, we analyze the number of samples in the dictionary and test the effectiveness of the elements that comprise the proposed model using various ablation studies.
- 12 December 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 316-319
I-vector based utterance verification for large-vocabulary speech recognition system
This paper proposes a new Utterance Verification (UV) algorithm based on i-vector. Phone segments are extracted and concatenated from the training data, which are used to train the Universal Background Model (UBM) and the Total Variability (TV) matrix, and then, i-vector is extracted from the enrollment and evaluation data using UBM and TV matrix. We compare two Confidence Measures (CMs), cosine distance scoring and Support Vector Machine (SVM). To compensate the channel effect, we use two channel compensation methods, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN). The decision is made by the word-level CM by combining the phone-level CMs. Experiments are conducted in the Korean isolated word recognition domain. Experimental results show that SVM is superior to cosine distance scoring. Best performance is achieved when SVM is used without any channel compensation method.
- 12 December 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 316-319
I-vector based utterance verification for large-vocabulary speech recognition system
This paper proposes a new Utterance Verification (UV) algorithm based on i-vector. Phone segments are extracted and concatenated from the training data, which are used to train the Universal Background Model (UBM) and the Total Variability (TV) matrix, and then, i-vector is extracted from the enrollment and evaluation data using UBM and TV matrix. We compare two Confidence Measures (CMs), cosine distance scoring and Support Vector Machine (SVM). To compensate the channel effect, we use two channel compensation methods, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN). The decision is made by the word-level CM by combining the phone-level CMs. Experiments are conducted in the Korean isolated word recognition domain. Experimental results show that SVM is superior to cosine distance scoring. Best performance is achieved when SVM is used without any channel compensation method.
- 1 April 2009
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15206149,p. 4365-4368
A new method for speaker adaptation using bilinear model
In this paper, a novel method for speaker adaptation using bilinear model is proposed. Bilinear model can express both characteristics of speakers (style) and phonemes across speakers (content) independently in a training database. The mapping from each speaker and phoneme space to observation space is carried out using bilinear mapping matrix which is independent of speaker and phoneme space. We apply the bilinear model to speaker adaption. Using adaptation data from a new speaker, speaker-adapted model is built by estimating the style(speaker)-specific matrix. Experimental results showed that the proposed method outperformed eigenvoice and MLLR. In vocabulary-independent isolated word recognition for speaker adaptation, bilinear model reduced word error rate by about 38% and about 10% compared to eigenvoice and MLLR respectively using 50 words for adaptation.
- 1 April 2009
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15206149,p. 4365-4368
A new method for speaker adaptation using bilinear model
In this paper, a novel method for speaker adaptation using bilinear model is proposed. Bilinear model can express both characteristics of speakers (style) and phonemes across speakers (content) independently in a training database. The mapping from each speaker and phoneme space to observation space is carried out using bilinear mapping matrix which is independent of speaker and phoneme space. We apply the bilinear model to speaker adaption. Using adaptation data from a new speaker, speaker-adapted model is built by estimating the style(speaker)-specific matrix. Experimental results showed that the proposed method outperformed eigenvoice and MLLR. In vocabulary-independent isolated word recognition for speaker adaptation, bilinear model reduced word error rate by about 38% and about 10% compared to eigenvoice and MLLR respectively using 50 words for adaptation.