Himanshu Kumar
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Himanshu Kumar has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Martin Aruldoss
Department of Computer Science, Central University of Tamil Nadu, Thiruvarur, India
5 shared publications A. Martin
1 shared publication- Martin Wynn
School of Business, Computing and Social Sciences, University of Gloucestershire, Cheltenham GL50 2RH, UK
1 shared publication
Latest Publications
- 19 May 2026
- Published by Springer Nature in Evolving Systems
- Vol. 17 (3) , 70
FG-T-FBP: fine-grained temporal factorized bilinear pooling fusion using graph attention mechanism for cross-modal emotion recognition
- 19 May 2026
- Published by Springer Nature in Evolving Systems
- Vol. 17 (3) , 70
FG-T-FBP: fine-grained temporal factorized bilinear pooling fusion using graph attention mechanism for cross-modal emotion recognition
- Open Access
- 24 November 2025
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 9 (12) , 116
Cross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition
Artificial emotional intelligence is a sub-domain of human–computer interaction research that aims to develop deep learning models capable of detecting and interpreting human emotional states through various modalities. A major challenge in this domain is identifying meaningful correlations between heterogeneous modalities—for example, between audio and visual data—due to their distinct temporal and spatial properties. Traditional fusion techniques used in multimodal learning to combine data from different sources often fail to adequately capture meaningful and less computational cross-modal interactions, and struggle to adapt to varying modality reliability. Following a review of the relevant literature, this study adopts an experimental research method to develop and evaluate a mathematical cross-modal fusion model, thereby addressing a gap in the extant research literature. The framework uses the Tucker tensor decomposition to analyse the multi-dimensional array of data into a set of matrices to support the integration of temporal features from audio and spatiotemporal features from visual modalities. A cross-attention mechanism is incorporated to enhance cross-modal interaction, enabling each modality to attend to the relevant information from the other. The efficacy of the model is rigorously evaluated on three publicly available datasets and the results conclusively demonstrate that the proposed fusion technique outperforms conventional fusion methods and several more recent approaches. The findings break new ground in this field of study and will be of interest to researchers and developers in artificial emotional intelligence.
- 24 November 2025
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 9 (12) , 116
Open AccessCross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition
Artificial emotional intelligence is a sub-domain of human–computer interaction research that aims to develop deep learning models capable of detecting and interpreting human emotional states through various modalities. A major challenge in this domain is identifying meaningful correlations between heterogeneous modalities—for example, between audio and visual data—due to their distinct temporal and spatial properties. Traditional fusion techniques used in multimodal learning to combine data from different sources often fail to adequately capture meaningful and less computational cross-modal interactions, and struggle to adapt to varying modality reliability. Following a review of the relevant literature, this study adopts an experimental research method to develop and evaluate a mathematical cross-modal fusion model, thereby addressing a gap in the extant research literature. The framework uses the Tucker tensor decomposition to analyse the multi-dimensional array of data into a set of matrices to support the integration of temporal features from audio and spatiotemporal features from visual modalities. A cross-attention mechanism is incorporated to enhance cross-modal interaction, enabling each modality to attend to the relevant information from the other. The efficacy of the model is rigorously evaluated on three publicly available datasets and the results conclusively demonstrate that the proposed fusion technique outperforms conventional fusion methods and several more recent approaches. The findings break new ground in this field of study and will be of interest to researchers and developers in artificial emotional intelligence.
- Open Access
- 3 April 2025
- Published by Engineering, Technology & Applied Science Research in Engineering, Technology & Applied Science Research
- Vol. 15 (2) , 20835-20841
Gated Cross-Modal Fusion Mechanism for Audio-Video-based Emotion Recognition
Due to its potential uses in security, surveillance, mental health monitoring, and human-computer interaction, artificial emotion recognition employing video and audio modalities has attracted a lot of attention. This study focuses on optimal cross-modal fusion techniques to enhance the precision and robustness of multimodal audio-video-based emotion recognition. Specifically, this study introduces a gated cross-modal fusion mechanism in audio-video-based emotion recognition, known as Compact Bilinear Gated Pooling (CBGP). The novelty of this work is that CBGP fusion is being applied to the emotion recognition task for the first time to integrate the extracted features and reduce the dimensionality of the audio and video modalities using 1DCNN and 3DCNN deep neural architectures, respectively. This novel approach was tested and verified on three benchmark datasets: CMU-MOSEI, RAVDESS, and IEMOCAP, each containing multimodal data representing a range of emotions, including happy, sad, fear, anger, neutral, and disgust. Experimental results show that CBGP consistently outperformed state-of-the-art fusion techniques, such as early fusion, late fusion, hybrid fusion, and others. CBGP extracts the relevant features, leading to higher accuracy and F1 scores due to its dynamic gating mechanism that selectively emphasizes relevant feature interactions. This study suggests that the integration of gating mechanisms within fusion processes is vital to improve emotion recognition. Future work will focus on extending these findings to real-time applications, exploring multitask learning frameworks, and enhancing the interpretability of multimodal emotion recognition systems.
- 3 April 2025
- Published by Engineering, Technology & Applied Science Research in Engineering, Technology & Applied Science Research
- Vol. 15 (2) , 20835-20841
Open AccessGated Cross-Modal Fusion Mechanism for Audio-Video-based Emotion Recognition
Due to its potential uses in security, surveillance, mental health monitoring, and human-computer interaction, artificial emotion recognition employing video and audio modalities has attracted a lot of attention. This study focuses on optimal cross-modal fusion techniques to enhance the precision and robustness of multimodal audio-video-based emotion recognition. Specifically, this study introduces a gated cross-modal fusion mechanism in audio-video-based emotion recognition, known as Compact Bilinear Gated Pooling (CBGP). The novelty of this work is that CBGP fusion is being applied to the emotion recognition task for the first time to integrate the extracted features and reduce the dimensionality of the audio and video modalities using 1DCNN and 3DCNN deep neural architectures, respectively. This novel approach was tested and verified on three benchmark datasets: CMU-MOSEI, RAVDESS, and IEMOCAP, each containing multimodal data representing a range of emotions, including happy, sad, fear, anger, neutral, and disgust. Experimental results show that CBGP consistently outperformed state-of-the-art fusion techniques, such as early fusion, late fusion, hybrid fusion, and others. CBGP extracts the relevant features, leading to higher accuracy and F1 scores due to its dynamic gating mechanism that selectively emphasizes relevant feature interactions. This study suggests that the integration of gating mechanisms within fusion processes is vital to improve emotion recognition. Future work will focus on extending these findings to real-time applications, exploring multitask learning frameworks, and enhancing the interpretability of multimodal emotion recognition systems.
- 16 October 2024
- Published by Edtech Publishers (OPC) Private Limited in International Journal of Scientific Research in Engineering and Management
- Vol. 08 (10) , 1-5
Cross-Modal Harmony: Low-Rank Fusion for Enhanced Artificial Emotion Recognition
Emotion recognition has become a subject of considerable interest in recent times, owing to its diverse and far-reaching applications in various fields. These applications span from enhancing human-computer interactions to assessing mental health and improving entertainment systems. The proposed study presents a novel approach for emotion recognition by fusing audio and video modalities using low-rank fusion techniques. The proposed methodology leverages the complementary nature of audio and video data in capturing emotional cues. Audio data often encapsulates tone, speech patterns, and vocal nuances, while video data captures facial expressions, body language, and gestures. However, the challenge lies in effectively integrating these two modalities to enhance recognition accuracy. To address the challenge, it employs low-rank fusion, a dimensionality reduction technique that extracts the most informative features from both modalities while minimizing redundancy. Furthermore, it presents the implementation of the chosen low-rank fusion algorithm in a real-world emotion recognition system. The results can contribute to advancing the field of emotion recognition by providing a practical and efficient solution for combining audio and video data to achieve more robust and accurate emotion classification. Keywords: Deep Learning; Emotion Recognition; Human-Computer Interaction; Low-Rank Fusion; Multimodal Fusion.
- 16 October 2024
- Published by Edtech Publishers (OPC) Private Limited in International Journal of Scientific Research in Engineering and Management
- Vol. 08 (10) , 1-5
Cross-Modal Harmony: Low-Rank Fusion for Enhanced Artificial Emotion Recognition
Emotion recognition has become a subject of considerable interest in recent times, owing to its diverse and far-reaching applications in various fields. These applications span from enhancing human-computer interactions to assessing mental health and improving entertainment systems. The proposed study presents a novel approach for emotion recognition by fusing audio and video modalities using low-rank fusion techniques. The proposed methodology leverages the complementary nature of audio and video data in capturing emotional cues. Audio data often encapsulates tone, speech patterns, and vocal nuances, while video data captures facial expressions, body language, and gestures. However, the challenge lies in effectively integrating these two modalities to enhance recognition accuracy. To address the challenge, it employs low-rank fusion, a dimensionality reduction technique that extracts the most informative features from both modalities while minimizing redundancy. Furthermore, it presents the implementation of the chosen low-rank fusion algorithm in a real-world emotion recognition system. The results can contribute to advancing the field of emotion recognition by providing a practical and efficient solution for combining audio and video data to achieve more robust and accurate emotion classification. Keywords: Deep Learning; Emotion Recognition; Human-Computer Interaction; Low-Rank Fusion; Multimodal Fusion.
- 16 April 2024
- Published by Springer Nature
- p. 213-225
Comparison and Performance Evaluation of Fusion Mechanism for Audio–Video Based Multimodal Emotion Recognition
- 16 April 2024
- Published by Springer Nature
- p. 213-225
Comparison and Performance Evaluation of Fusion Mechanism for Audio–Video Based Multimodal Emotion Recognition
- 24 August 2022
- Published by Elsevier in Expert Systems with Applications
- Vol. 212
Artificial Emotional Intelligence: Conventional and deep learning approach
- 24 August 2022
- Published by Elsevier in Expert Systems with Applications
- Vol. 212
Artificial Emotional Intelligence: Conventional and deep learning approach