HSHwa Jeon Song

Hwa Jeon Song

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Hwa Jeon Song has a total of 20 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Modeling Long-Term Multimodal Representations for Active Speaker Detection With Spatio-Positional Encoder

    by Minyoung Kyoung, Hwa Jeon Song

    In this study, we present an end-to-end framework for active speaker detection to achieve robust performance in challenging scenarios with multiple speakers. In contrast to recent approaches, which rely heavily on the visual relational context between all speakers in a video frame, we propose collaboratively learning multimodal representations based on the audio and visual signals of a single candidate. Firstly, we propose a spatio-positional encoder to effectively address the problem of false detections caused by indiscernible faces in a video frame. Secondly, we present an efficient multimodal approach that models the long-term temporal contextual interactions between audio and visual modalities. Through extensive experiments on the AVA-ActiveSpeaker dataset, we demonstrate that our framework notably outperforms recent state-of-the-art approaches under challenging multi-speaker settings. Additionally, the proposed framework significantly improves the robustness against auditory and visual noise interference without relying on pre-trained networks or hand-crafted training strategies.


  • Open Access

    CuMARL: Curiosity-Based Learning in Multiagent Reinforcement Learning

    by Devarani Devi Ningombam, Byunghyun Yoo, Hyun Woo Kim, Hwa Jeon Song, Sungwon Yi

    In this paper, we propose a novel curiosity-based learning algorithm for Multi-agent Reinforcement Learning (MARL) to attain efficient and effective decision-making. We employ the centralized training with decentralized execution framework (CTDE) and consider that each agent has knowledge of the prior action distribution of others. To quantify the difference in agents’ knowledge, curiosity, we introduce conditional mutual information (CMI) regularization and use the amount of information for updating decision-making policy. Then, to deploy these learning frameworks in a large-scale MARL setting while acquiring high sample efficiency, we consider a Kullback-Leibler (KL) divergence-based prioritization of experiences. We evaluate the effectiveness of the proposed algorithm in three different levels of StarCraft Multi Agent Challenge (SMAC) scenarios using the PyMARL framework. The simulation-based performance analysis shows that the proposed technique significantly improves the test win rate compared to various state-of-the-art MARL benchmarks, such as the Optimistically Weighted Monotonic Value Function Factorization (OW_QMIX) and Learning Individual Intrinsic Reward (LIIR).


  • Open Access

    A Novel and Efficient Influence-Seeking Exploration in Deep Multiagent Reinforcement Learning

    by Byunghyun Yoo, Devarani Devi Ningombam, Sungwon Yi, Hyun Woo Kim, Euisok Chung, Ran Han, Hwa Jeon Song

    Although recent years witnessed notable success for a cooperative setting in multi-agent reinforcement learning (MARL), efficient explorations are still challenging primarily due to the complex dynamics of inter-agent interactions constituting the high dimension of action spaces. For an efficient exploration, it is necessary to quantify influences that can represent interactions among agents and use them to obtain more information about the complexity of multi-agent systems. In this paper, we propose a novel influence-seeking exploration (ISE) scheme, which encourages agents to preferably explore action spaces significantly influenced by others and thus helps in speeding up the learning curve. To measure the influence of other agents in action selection, we use the variance of joint action-values with different action sets of agents that obtained by an estimation technique to lessen computation overhead. To this end, we first present an analytical approach inspired by the concept of approximated variance propagation and then apply it to an exploration scheme. We evaluate the proposed exploration method on a set of StarCraft II micromanagement as well as modified predator-prey tasks. Compared to state-of-the-art methods, the proposed method achieved performance improvements of 10% in StarCraft II micromanagement and 50% in modified predator-prey tasks approximately.


  • Open Access

    Imbalanced Classification via Feature Dictionary-Based Minority Oversampling

    by Minho Park, Hwa Jeon Song, Dong-Oh Kang

    Image classification research is one of the fields continuously studied in the computer vision domain, and several related studies have been actively conducted until recently. However, a limit exists regarding the prediction performance of real-world datasets due to the data imbalance problem between classes. Data augmentation through artificial sample generation for minority classes is one of the methods used to overcome this limitation. Among the various oversampling methods, we propose the feature dictionary-based generative model for the oversampling method. Feature dictionaries are built through the pretrained feature extractor, and the proposed generative model synthesizes artificial samples based on the dictionary. Class-to-class balanced training can be conducted by fine-tuning the classifier as additional data for the minority class. We experiment by applying the proposed framework to the fashion dataset, which has an extreme class imbalance. The experimental results demonstrate that the proposed model achieved the highest top-1 performance on various public fashion datasets. In addition, we analyze the number of samples in the dictionary and test the effectiveness of the elements that comprise the proposed model using various ablation studies.


  • I-vector based utterance verification for large-vocabulary speech recognition system

    by Woo Yong Choi, Hwa Jeon Song, Hoon Chung, Jeomja Kang, Jeon Gue Park

    This paper proposes a new Utterance Verification (UV) algorithm based on i-vector. Phone segments are extracted and concatenated from the training data, which are used to train the Universal Background Model (UBM) and the Total Variability (TV) matrix, and then, i-vector is extracted from the enrollment and evaluation data using UBM and TV matrix. We compare two Confidence Measures (CMs), cosine distance scoring and Support Vector Machine (SVM). To compensate the channel effect, we use two channel compensation methods, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN). The decision is made by the word-level CM by combining the phone-level CMs. Experiments are conducted in the Korean isolated word recognition domain. Experimental results show that SVM is superior to cosine distance scoring. Best performance is achieved when SVM is used without any channel compensation method.


  • A new method for speaker adaptation using bilinear model

    by Hwa Jeon Song, Yongwon Jeong, Hyung Soon Kim

    In this paper, a novel method for speaker adaptation using bilinear model is proposed. Bilinear model can express both characteristics of speakers (style) and phonemes across speakers (content) independently in a training database. The mapping from each speaker and phoneme space to observation space is carried out using bilinear mapping matrix which is independent of speaker and phoneme space. We apply the bilinear model to speaker adaption. Using adaptation data from a new speaker, speaker-adapted model is built by estimating the style(speaker)-specific matrix. Experimental results showed that the proposed method outperformed eigenvoice and MLLR. In vocabulary-independent isolated word recognition for speaker adaptation, bilinear model reduced word error rate by about 38% and about 10% compared to eigenvoice and MLLR respectively using 50 words for adaptation.