Gangshan Wu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Gangshan Wu has a total of 241 co-authors. Here are the top co-authors by number of shared publications:
- Limin Wang
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
28 shared publications - Jie Tang
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
25 shared publications - Jie Liu
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
24 shared publications - Tongwei Ren
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
14 shared publications - Ruichao Hou
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
12 shared publications - Boyue Xu
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
7 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7969-7978
Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7969-7978
Open AccessEnabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
- 9 October 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (10) , 468
Efficient Test-Time Prompt Tuning for Vision-Language Models
- 9 October 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (10) , 468
Efficient Test-Time Prompt Tuning for Vision-Language Models
- Open Access
- 18 March 2026
- Published by Elsevier in SSRN Electronic Journal
A Training-Free Solution for Adjustable Portrait Animation with Flexible Mixed-Modal Controls
Portrait animation synthesizes lifelike videos for diverse commercial applications, utilizing widely available internet-sourced multi-modal inputs, including reference images, audio, and motion video sequences. Most previous methods usually need to train specialized models under specific conditions, which is computationally expensive for real-world applications. In this paper, we introduce a training-free, adjustable portrait animation framework that can be extended to diverse trained portrait animation models without additional training. This framework allows for various combinations of mixed-modal controls for more controllable portrait video generation. By preserving or changing different driving conditions, the framework can flexibly support the three parts of the generated video, changed or remained - the portrait identity, lip movements, and head pose. Specifically, our method is based on a Unified Animation Control (UAC) mechanism with DDIM inversion latents from the driven video, including visual-driven shape control, audio-driven speaking control, and inter-frame temporal control. We mainly compare our generation results with lip reenactment methods and portrait replacement methods. The experimental results show that our model can achieve more accurate and synchronized lip movements, as well as more flexible motion transfer with head rotations and facial expressions. Moreover, by modifying the initial reference image, the framework enables stylization, background alterations, head rotations, and expression adjustments, ensuring its adaptability to a wider range of scenarios.
- 18 March 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessA Training-Free Solution for Adjustable Portrait Animation with Flexible Mixed-Modal Controls
Portrait animation synthesizes lifelike videos for diverse commercial applications, utilizing widely available internet-sourced multi-modal inputs, including reference images, audio, and motion video sequences. Most previous methods usually need to train specialized models under specific conditions, which is computationally expensive for real-world applications. In this paper, we introduce a training-free, adjustable portrait animation framework that can be extended to diverse trained portrait animation models without additional training. This framework allows for various combinations of mixed-modal controls for more controllable portrait video generation. By preserving or changing different driving conditions, the framework can flexibly support the three parts of the generated video, changed or remained - the portrait identity, lip movements, and head pose. Specifically, our method is based on a Unified Animation Control (UAC) mechanism with DDIM inversion latents from the driven video, including visual-driven shape control, audio-driven speaking control, and inter-frame temporal control. We mainly compare our generation results with lip reenactment methods and portrait replacement methods. The experimental results show that our model can achieve more accurate and synchronized lip movements, as well as more flexible motion transfer with head rotations and facial expressions. Moreover, by modifying the initial reference image, the framework enables stylization, background alterations, head rotations, and expression adjustments, ensuring its adaptability to a wider range of scenarios.
- Top Cited Papers
- 18 March 2026
- Published by Elsevier in Pattern Recognition
- Vol. 179
Learning frequency and memory-aware prompts for multi-modal object tracking
- 18 March 2026
- Published by Elsevier in Pattern Recognition
- Vol. 179
Top Cited PapersLearning frequency and memory-aware prompts for multi-modal object tracking
- 15 February 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (3) , 121
GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates
- 15 February 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (3) , 121
GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates
- Open Access
GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates
Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD
Open AccessGLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates
Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD
- 13 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33 (10709908) , 594-598
Cross-View and Cross-Modal Contrastive Learning for Radar Object Detection
Frequency-modulated continuous-wave radar is a cornerstone of advanced driver assistance systems thanks to its low cost and resilience to adverse weather. Yet the absence of explicit semantics makes radar annotation difficult, and the scarcity of large-scale labeled data limits the performance of radar perception models. To address this issue, we propose a self-supervised framework for object detection directly from Range–Azimuth– Doppler (RAD) cubes that learns transferable representations from unlabeled radar data. Specifically, we introduce cross-view contrastive learning to model correspondences among complementary views of the RAD cube, encouraging the network to capture spatial structure from multiple perspectives. In addition, an auxiliary cross-modal contrastive objective distills semantic knowledge from vision into radar. The joint objective integrates cross-view and cross-modal signals to strengthen radar feature representations. We further extend the framework to cross-domain pretraining using datasets from different sources. Experimental results demonstrate that the proposed method significantly improves radar object detection performance, especially with limited labeled data.
- 13 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33 (10709908) , 594-598
Cross-View and Cross-Modal Contrastive Learning for Radar Object Detection
Frequency-modulated continuous-wave radar is a cornerstone of advanced driver assistance systems thanks to its low cost and resilience to adverse weather. Yet the absence of explicit semantics makes radar annotation difficult, and the scarcity of large-scale labeled data limits the performance of radar perception models. To address this issue, we propose a self-supervised framework for object detection directly from Range–Azimuth– Doppler (RAD) cubes that learns transferable representations from unlabeled radar data. Specifically, we introduce cross-view contrastive learning to model correspondences among complementary views of the RAD cube, encouraging the network to capture spatial structure from multiple perspectives. In addition, an auxiliary cross-modal contrastive objective distills semantic knowledge from vision into radar. The joint objective integrates cross-view and cross-modal signals to strengthen radar feature representations. We further extend the framework to cross-domain pretraining using datasets from different sources. Experimental results demonstrate that the proposed method significantly improves radar object detection performance, especially with limited labeled data.
- Open Access
Towards Practical Real-Time Low-Latency Music Source Separation
In recent years, significant progress has been made in the field of deep learning for music demixing. However, there has been limited attention on real-time, low-latency music demixing, which holds potential for various applications, such as hearing aids, audio stream remixing, and live performances. Additionally, a notable tendency has emerged towards the development of larger models, limiting their applicability in certain scenarios. In this paper, we introduce a lightweight real-time low-latency model called Real-Time Single-Path TFC-TDF UNET (RT-STT), which is based on the Dual-Path TFC-TDF UNET (DTTNet). In RT-STT, we propose a feature fusion technique based on channel expansion. We also demonstrate the superiority of single-path modeling over dual-path modeling in real-time models. Moreover, we investigate the method of quantization to further reduce inference time. RT-STT exhibits superior performance with significantly fewer parameters and shorter inference times compared to state-of-the-art models.
Open AccessTowards Practical Real-Time Low-Latency Music Source Separation
In recent years, significant progress has been made in the field of deep learning for music demixing. However, there has been limited attention on real-time, low-latency music demixing, which holds potential for various applications, such as hearing aids, audio stream remixing, and live performances. Additionally, a notable tendency has emerged towards the development of larger models, limiting their applicability in certain scenarios. In this paper, we introduce a lightweight real-time low-latency model called Real-Time Single-Path TFC-TDF UNET (RT-STT), which is based on the Dual-Path TFC-TDF UNET (DTTNet). In RT-STT, we propose a feature fusion technique based on channel expansion. We also demonstrate the superiority of single-path modeling over dual-path modeling in real-time models. Moreover, we investigate the method of quantization to further reduce inference time. RT-STT exhibits superior performance with significantly fewer parameters and shorter inference times compared to state-of-the-art models.
- 24 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (3) , 2697-2712
HyPSAM: Hybrid Prompt-Driven Segment Anything Model for RGB-Thermal Salient Object Detection
RGB-thermal salient object detection (RGB-T SOD) aims to identify prominent objects by integrating complementary information from RGB and thermal modalities. However, learning the precise boundaries and complete objects remains challenging due to the intrinsic insufficient feature fusion and the extrinsic limitations of data scarcity. In this paper, we propose a novel hybrid prompt-driven segment anything model (HyPSAM), which leverages the zero-shot generalization capabilities of the segment anything model (SAM) for RGB-T SOD. Specifically, we first propose a dynamic fusion network (DFNet) that generates high-quality initial saliency maps as visual prompts. DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction, overcoming the limitations of fixed-parameter kernels and enhancing multi-modal feature representation. Moreover, we propose a plug-and-play refinement network (P2RNet) which serves as a general optimization strategy to guide SAM in refining saliency maps by using hybrid prompts. The text prompt ensures reliable modality input, while the mask and box prompts enable precise salient object localization. Extensive experiments on three public datasets demonstrate that our method achieves state-of-the-art performance. Notably, HyPSAM has remarkable versatility, seamlessly integrating with different RGB-T SOD methods to achieve significant performance gains, thereby highlighting the potential of prompt engineering in this field. The code and results of our method are available at: https://github.com/milotic233/HyPSAM.
- 24 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (3) , 2697-2712
HyPSAM: Hybrid Prompt-Driven Segment Anything Model for RGB-Thermal Salient Object Detection
RGB-thermal salient object detection (RGB-T SOD) aims to identify prominent objects by integrating complementary information from RGB and thermal modalities. However, learning the precise boundaries and complete objects remains challenging due to the intrinsic insufficient feature fusion and the extrinsic limitations of data scarcity. In this paper, we propose a novel hybrid prompt-driven segment anything model (HyPSAM), which leverages the zero-shot generalization capabilities of the segment anything model (SAM) for RGB-T SOD. Specifically, we first propose a dynamic fusion network (DFNet) that generates high-quality initial saliency maps as visual prompts. DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction, overcoming the limitations of fixed-parameter kernels and enhancing multi-modal feature representation. Moreover, we propose a plug-and-play refinement network (P2RNet) which serves as a general optimization strategy to guide SAM in refining saliency maps by using hybrid prompts. The text prompt ensures reliable modality input, while the mask and box prompts enable precise salient object localization. Extensive experiments on three public datasets demonstrate that our method achieves state-of-the-art performance. Notably, HyPSAM has remarkable versatility, seamlessly integrating with different RGB-T SOD methods to achieve significant performance gains, thereby highlighting the potential of prompt engineering in this field. The code and results of our method are available at: https://github.com/milotic233/HyPSAM.