Jinjing Gu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jinjing Gu has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
Zhengpeng Zhao
6 shared publications- Yuanyuan Pu
Yunnan University
5 shared publications Yuanyuan Pu
4 shared publicationsChenggang Yang
3 shared publications- Zhengpeng Zhao
School of Information Science and Engineering, Yunnan University, Kunming, China
3 shared publications - Dan Xu
Yunnan University
2 shared publications
Latest Publications
- 9 September 2026
- Published by Elsevier in Image and Vision Computing
- Vol. 175
HO-GCN: Human–Object Adaptive Topological Interaction GCN for escalator dangerous behavior recognition
- 9 September 2026
- Published by Elsevier in Image and Vision Computing
- Vol. 175
HO-GCN: Human–Object Adaptive Topological Interaction GCN for escalator dangerous behavior recognition
- Open Access
- 8 September 2026
- Published by Elsevier in SSRN Electronic Journal
Temporal-frequency assisted video-text alignment for weakly supervised video anomaly detection
Weakly supervised video anomaly detection~(WSVAD) aims to identify abnormal events in videos with only coarse video-level annotations. Previous works mainly focused on modeling anomalies in spatio-temporal domains of visual modality. However, how to capture the global frame-level video features and fully utilize the anchoring role of textual features still remains challenges. To address this dilemma, a novel TV-VAD framework that employs Temporal-frequency assisted Video-text alignment for WSVAD is proposed. In the temporal-frequency video feature extraction module, the transformer encoder combined with a dual-branch graph convolution neural module is used to model the temporal dynamics within videos, and a Fourier frequency-domain feature recombination module is constructed to capture the varied expressions of anomalies. Then, the Video-Text alignment block with Kullback-Leibler divergence is designed to further align the similarity between the extracted global frame-level video features and the fused video and textual features, which is helpful to fully leverage the anchoring role of the text modality. Furthermore, a multiclass loss function is introduced to transform the traditional binary classification task of WSVAD into a multiclass classification task. Extensive experiments show that the proposed TV-VAD model achieves state-of-the-art performance on XD-Violence with an AP of 85.61%, and UCF-Crime, where AUC reaches up to 88.12%. Code is available at: https://github.com/jinjinggu00/TV-VAD.
- 8 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessTemporal-frequency assisted video-text alignment for weakly supervised video anomaly detection
Weakly supervised video anomaly detection~(WSVAD) aims to identify abnormal events in videos with only coarse video-level annotations. Previous works mainly focused on modeling anomalies in spatio-temporal domains of visual modality. However, how to capture the global frame-level video features and fully utilize the anchoring role of textual features still remains challenges. To address this dilemma, a novel TV-VAD framework that employs Temporal-frequency assisted Video-text alignment for WSVAD is proposed. In the temporal-frequency video feature extraction module, the transformer encoder combined with a dual-branch graph convolution neural module is used to model the temporal dynamics within videos, and a Fourier frequency-domain feature recombination module is constructed to capture the varied expressions of anomalies. Then, the Video-Text alignment block with Kullback-Leibler divergence is designed to further align the similarity between the extracted global frame-level video features and the fused video and textual features, which is helpful to fully leverage the anchoring role of the text modality. Furthermore, a multiclass loss function is introduced to transform the traditional binary classification task of WSVAD into a multiclass classification task. Extensive experiments show that the proposed TV-VAD model achieves state-of-the-art performance on XD-Violence with an AP of 85.61%, and UCF-Crime, where AUC reaches up to 88.12%. Code is available at: https://github.com/jinjinggu00/TV-VAD.
- 20 July 2026
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 181
Adverse multi-weather image restoration for boosting downstream object detection
- 20 July 2026
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 181
Adverse multi-weather image restoration for boosting downstream object detection
- Open Access
- 15 April 2026
- Published by Elsevier in SSRN Electronic Journal
TV-VAD: Temporal-frequency assisted video-text alignment for weakly supervised video anomaly detection
Weakly supervised video anomaly detection (WSVAD) aims to identify abnormal events in videos with only coarse video-level annotations. Previous methods primarily focus on modeling anomalies within the spatio-temporal domain of visual modality. However, how to capture global frame-level video features and fully utilize the anchoring role of textual features remains challenging. To address these dilemmas, a novel TV-VAD framework that employs Temporal-frequency assisted Video-text alignment for WSVAD is proposed. Specifically, the temporal-frequency video feature extraction module integrates a Transformer encoder with a dual-branch graph convolutional network to model temporal dynamics, while a Fourier frequency-domain feature recombination module is introduced to capture diverse expressions of anomalies. Subsequently, a video-text alignment block based on Kullback-Leibler divergence aligns the global frame-level video features with fused multi-modal representations, thereby better exploiting the anchoring role of the text modality. Furthermore, a multiclass loss function is employed to reformulate the traditional binary classification task of WSVAD into a multiclass classification task. Extensive experiments demonstrate that TV-VAD achieves state-of-the-art performance on two public benchmarks, achieving an AP of 85.61% on XD-Violence and an AUC of 88.12% on UCF-Crime. Code is available at: https://github.com/jinjinggu00/TV-VAD.
- 15 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessTV-VAD: Temporal-frequency assisted video-text alignment for weakly supervised video anomaly detection
Weakly supervised video anomaly detection (WSVAD) aims to identify abnormal events in videos with only coarse video-level annotations. Previous methods primarily focus on modeling anomalies within the spatio-temporal domain of visual modality. However, how to capture global frame-level video features and fully utilize the anchoring role of textual features remains challenging. To address these dilemmas, a novel TV-VAD framework that employs Temporal-frequency assisted Video-text alignment for WSVAD is proposed. Specifically, the temporal-frequency video feature extraction module integrates a Transformer encoder with a dual-branch graph convolutional network to model temporal dynamics, while a Fourier frequency-domain feature recombination module is introduced to capture diverse expressions of anomalies. Subsequently, a video-text alignment block based on Kullback-Leibler divergence aligns the global frame-level video features with fused multi-modal representations, thereby better exploiting the anchoring role of the text modality. Furthermore, a multiclass loss function is employed to reformulate the traditional binary classification task of WSVAD into a multiclass classification task. Extensive experiments demonstrate that TV-VAD achieves state-of-the-art performance on two public benchmarks, achieving an AP of 85.61% on XD-Violence and an AUC of 88.12% on UCF-Crime. Code is available at: https://github.com/jinjinggu00/TV-VAD.
- 20 March 2026
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 174
Retrieval-based objects and relations prompt for image captioning
- 20 March 2026
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 174
Retrieval-based objects and relations prompt for image captioning
- 28 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 17 (1) , 856-868
CoMPLe: Cross-Modal Hybrid Prompt Learning for End-to-End Multimodal Emotion Recognition
The quality of features directly affects the accuracy of Multimodal Emotion Recognition (MER). A key challenge in this context is the effective extraction of dynamically interactive multimodal features to enrich conversational emotion representations. However, existing approaches are often constrained by non-end-to-end architectures, overlooking the significance of feature extraction in MER. To address the problem of dynamic interaction in emotion feature extraction, this paper introduces an end-to-end network based on Cross-Modal Hybrid Prompt Learning (CoMPLe). The model takes raw video as input and leverages three prompt mechanisms to guide large-scale pre-trained encoders in extracting emotionally salient features with latent correlations. Specifically, we design a cross-modal soft prompt learning strategy to mine complementary information across modalities and dynamically adjust the cross-modal semantic space. To capture stage-dependent characteristics, deep feature prompts are incorporated to progressively learn intra-modal contextual representations. Furthermore, a label prompt mechanism is proposed to construct hard prompt templates from emotion labels. Finally, the highest cosine similarity is computed between each unimodal feature and the label prompt templates to activate factual knowledge relevant to emotion recognition. Experiments on three public datasets show that the end-to-end network proposed in this paper surpasses the existing State-Of-The-Art baselines.
- 28 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 17 (1) , 856-868
CoMPLe: Cross-Modal Hybrid Prompt Learning for End-to-End Multimodal Emotion Recognition
The quality of features directly affects the accuracy of Multimodal Emotion Recognition (MER). A key challenge in this context is the effective extraction of dynamically interactive multimodal features to enrich conversational emotion representations. However, existing approaches are often constrained by non-end-to-end architectures, overlooking the significance of feature extraction in MER. To address the problem of dynamic interaction in emotion feature extraction, this paper introduces an end-to-end network based on Cross-Modal Hybrid Prompt Learning (CoMPLe). The model takes raw video as input and leverages three prompt mechanisms to guide large-scale pre-trained encoders in extracting emotionally salient features with latent correlations. Specifically, we design a cross-modal soft prompt learning strategy to mine complementary information across modalities and dynamically adjust the cross-modal semantic space. To capture stage-dependent characteristics, deep feature prompts are incorporated to progressively learn intra-modal contextual representations. Furthermore, a label prompt mechanism is proposed to construct hard prompt templates from emotion labels. Finally, the highest cosine similarity is computed between each unimodal feature and the label prompt templates to activate factual knowledge relevant to emotion recognition. Experiments on three public datasets show that the end-to-end network proposed in this paper surpasses the existing State-Of-The-Art baselines.
- 25 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1228-1241
Leveraging Enriched Skeleton Representation With Multi-Relational Metrics for Few-Shot Action Recognition
Few-shot action recognition aims to identify new action classes with limited training samples. Most existing methods overlook the low information content and diversity of skeleton features, failing to exploit useful information in rare samples during meta-training. This leads to poor feature discriminability and recognition accuracy. To address both issues, we propose a novel Enriched Skeleton Representation and Multirelational Metrics (ESR-MM) method for skeleton-based few-shot action recognition. First, a Frobenius Norm Diversity Loss is introduced to enrich skeleton representation by maximizing the Frobenius norm of the skeleton feature matrix. This mitigates over-smoothing and boosts information content and diversity. Leveraging these enriched features, we propose a multi-relational metrics strategy exploiting cross-sample task-specific information, intra-sample temporal order, and inter-sample distance. Specifically, Support-Adaptive Attention leverages task-specific cues between samples to generate attention-enhanced features. Then, the Bidirectional Temporal Coherent Mean Hausdorff Metric integrates Temporal Coherence Measure into the Bidirectional Mean Hausdorff Metric for class separation by accounting for temporal order. Finally, Prototype-discriminative Contrastive Loss exploits distances from class prototypes to query samples. ESR-MM demonstrates superior performance on two benchmarks. To support reproducibility and further research, our code is available at https://github.com/jinjinggu00/ESR-MM .
- 25 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1228-1241
Leveraging Enriched Skeleton Representation With Multi-Relational Metrics for Few-Shot Action Recognition
Few-shot action recognition aims to identify new action classes with limited training samples. Most existing methods overlook the low information content and diversity of skeleton features, failing to exploit useful information in rare samples during meta-training. This leads to poor feature discriminability and recognition accuracy. To address both issues, we propose a novel Enriched Skeleton Representation and Multirelational Metrics (ESR-MM) method for skeleton-based few-shot action recognition. First, a Frobenius Norm Diversity Loss is introduced to enrich skeleton representation by maximizing the Frobenius norm of the skeleton feature matrix. This mitigates over-smoothing and boosts information content and diversity. Leveraging these enriched features, we propose a multi-relational metrics strategy exploiting cross-sample task-specific information, intra-sample temporal order, and inter-sample distance. Specifically, Support-Adaptive Attention leverages task-specific cues between samples to generate attention-enhanced features. Then, the Bidirectional Temporal Coherent Mean Hausdorff Metric integrates Temporal Coherence Measure into the Bidirectional Mean Hausdorff Metric for class separation by accounting for temporal order. Finally, Prototype-discriminative Contrastive Loss exploits distances from class prototypes to query samples. ESR-MM demonstrates superior performance on two benchmarks. To support reproducibility and further research, our code is available at https://github.com/jinjinggu00/ESR-MM .
- 2 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-12
Skeleton-Based Few-Shot Action Recognition via Fine-Grained Information Capture and Adaptive Metric Aggregation
Recognizing novel actions based on few labeled skeleton sequence samples is a promising field. Existing works primarily focus on global representations of similar actions and designing novel metric functions. This approach overlooks fine-grained and unique characteristics within samples, which are crucial for distinguishing subtle differences between actions. Moreover, as the field advances, designing increasingly complex metric functions yields diminishing returns in performance improvements. To address these issues, we propose the Fine-grained Information Capture and Adaptive Metric Aggregation (FICAMA) framework for skeleton-based few-shot action recognition. This framework enhances the self-information of support samples to facilitate the capture of fine-grained representations. The combination of fine-grained representations and coarse-grained global representations contributes to better action matching. We then employ the Skeletal Motion Fusion Module to spatially combine coarse-grained and fine-grained representations within samples and introduce temporal context information through positional encoding. Additionally, we design a Adaptive Multi-Metric Distance Aggregation Module capable of simultaneously utilizing multiple metric functions to aggregate the results of different metric functions in a task-adaptive manner and achieve high-precision skeleton sequence matching. Experiments on the NTU RGB+D 120 and Kinetics datasets demonstrate the effectiveness of our approach. Code is available at https://github.com/jinjinggu00/FICAMA.
- 2 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-12
Skeleton-Based Few-Shot Action Recognition via Fine-Grained Information Capture and Adaptive Metric Aggregation
Recognizing novel actions based on few labeled skeleton sequence samples is a promising field. Existing works primarily focus on global representations of similar actions and designing novel metric functions. This approach overlooks fine-grained and unique characteristics within samples, which are crucial for distinguishing subtle differences between actions. Moreover, as the field advances, designing increasingly complex metric functions yields diminishing returns in performance improvements. To address these issues, we propose the Fine-grained Information Capture and Adaptive Metric Aggregation (FICAMA) framework for skeleton-based few-shot action recognition. This framework enhances the self-information of support samples to facilitate the capture of fine-grained representations. The combination of fine-grained representations and coarse-grained global representations contributes to better action matching. We then employ the Skeletal Motion Fusion Module to spatially combine coarse-grained and fine-grained representations within samples and introduce temporal context information through positional encoding. Additionally, we design a Adaptive Multi-Metric Distance Aggregation Module capable of simultaneously utilizing multiple metric functions to aggregate the results of different metric functions in a task-adaptive manner and achieve high-precision skeleton sequence matching. Experiments on the NTU RGB+D 120 and Kinetics datasets demonstrate the effectiveness of our approach. Code is available at https://github.com/jinjinggu00/FICAMA.
- 8 November 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 263
FNContra: Frequency-domain Negative Sample Mining in Contrastive Learning for limited-data image generation
- 8 November 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 263
FNContra: Frequency-domain Negative Sample Mining in Contrastive Learning for limited-data image generation