GWGangshan Wu

Gangshan Wu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Gangshan Wu has a total of 241 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework

    by Tianrui Pan, Qinglin Zhang, Chong Deng, Luyao Cheng, Qian Chen, Wen Wang, Jie Tang, Gangshan Wu, Jie Liu

  • Efficient Test-Time Prompt Tuning for Vision-Language Models

    by Yuhan Zhu, Guozhen Zhang, Chen Xu, Haocheng Shen, Xiaoxin Chen, Gangshan Wu, Limin Wang

  • Open Access

    A Training-Free Solution for Adjustable Portrait Animation with Flexible Mixed-Modal Controls

    by Tianrui Pan, Lin Liu, Jie Liu, Xiaopeng Zhang, Jie Tang, Gangshan Wu, Qi Tian

    Portrait animation synthesizes lifelike videos for diverse commercial applications, utilizing widely available internet-sourced multi-modal inputs, including reference images, audio, and motion video sequences. Most previous methods usually need to train specialized models under specific conditions, which is computationally expensive for real-world applications. In this paper, we introduce a training-free, adjustable portrait animation framework that can be extended to diverse trained portrait animation models without additional training. This framework allows for various combinations of mixed-modal controls for more controllable portrait video generation. By preserving or changing different driving conditions, the framework can flexibly support the three parts of the generated video, changed or remained - the portrait identity, lip movements, and head pose. Specifically, our method is based on a Unified Animation Control (UAC) mechanism with DDIM inversion latents from the driven video, including visual-driven shape control, audio-driven speaking control, and inter-frame temporal control. We mainly compare our generation results with lip reenactment methods and portrait replacement methods. The experimental results show that our model can achieve more accurate and synchronized lip movements, as well as more flexible motion transfer with head rotations and facial expressions. Moreover, by modifying the initial reference image, the framework enables stylization, background alterations, head rotations, and expression adjustments, ensuring its adaptability to a wider range of scenarios.

  • Top Cited Papers

    Learning frequency and memory-aware prompts for multi-modal object tracking

    by Boyue Xu, Ruichao Hou, Tongwei Ren, Dongming Zhou, Gangshan Wu, Jinde Cao


  • GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates

    by Xingyu Luo, Yidong Cai, Jie Liu, Jie Tang, Gangshan Wu, Limin Wang


  • Open Access

    GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates

    by Xingyu Luo, Yidong Cai, Jie Liu, Jie Tang, Gangshan Wu, Limin Wang

    Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD

  • Cross-View and Cross-Modal Contrastive Learning for Radar Object Detection

    by Qiaolong Qian, Yi Shi, Ruichao Hou, Haoyu Qin, Gangshan Wu

    Frequency-modulated continuous-wave radar is a cornerstone of advanced driver assistance systems thanks to its low cost and resilience to adverse weather. Yet the absence of explicit semantics makes radar annotation difficult, and the scarcity of large-scale labeled data limits the performance of radar perception models. To address this issue, we propose a self-supervised framework for object detection directly from Range–Azimuth– Doppler (RAD) cubes that learns transferable representations from unlabeled radar data. Specifically, we introduce cross-view contrastive learning to model correspondences among complementary views of the RAD cube, encouraging the network to capture spatial structure from multiple perspectives. In addition, an auxiliary cross-modal contrastive objective distills semantic knowledge from vision into radar. The joint objective integrates cross-view and cross-modal signals to strengthen radar feature representations. We further extend the framework to cross-domain pretraining using datasets from different sources. Experimental results demonstrate that the proposed method significantly improves radar object detection performance, especially with limited labeled data.


  • Open Access

    Towards Practical Real-Time Low-Latency Music Source Separation

    by Junyu Wu, Jie Liu, Tianrui Pan, Jie Tang, Gangshan Wu

    In recent years, significant progress has been made in the field of deep learning for music demixing. However, there has been limited attention on real-time, low-latency music demixing, which holds potential for various applications, such as hearing aids, audio stream remixing, and live performances. Additionally, a notable tendency has emerged towards the development of larger models, limiting their applicability in certain scenarios. In this paper, we introduce a lightweight real-time low-latency model called Real-Time Single-Path TFC-TDF UNET (RT-STT), which is based on the Dual-Path TFC-TDF UNET (DTTNet). In RT-STT, we propose a feature fusion technique based on channel expansion. We also demonstrate the superiority of single-path modeling over dual-path modeling in real-time models. Moreover, we investigate the method of quantization to further reduce inference time. RT-STT exhibits superior performance with significantly fewer parameters and shorter inference times compared to state-of-the-art models.

  • HyPSAM: Hybrid Prompt-Driven Segment Anything Model for RGB-Thermal Salient Object Detection

    by Ruichao Hou, Xingyuan Li, Tongwei Ren, Dongming Zhou, Gangshan Wu, Jinde Cao

    RGB-thermal salient object detection (RGB-T SOD) aims to identify prominent objects by integrating complementary information from RGB and thermal modalities. However, learning the precise boundaries and complete objects remains challenging due to the intrinsic insufficient feature fusion and the extrinsic limitations of data scarcity. In this paper, we propose a novel hybrid prompt-driven segment anything model (HyPSAM), which leverages the zero-shot generalization capabilities of the segment anything model (SAM) for RGB-T SOD. Specifically, we first propose a dynamic fusion network (DFNet) that generates high-quality initial saliency maps as visual prompts. DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction, overcoming the limitations of fixed-parameter kernels and enhancing multi-modal feature representation. Moreover, we propose a plug-and-play refinement network (P2RNet) which serves as a general optimization strategy to guide SAM in refining saliency maps by using hybrid prompts. The text prompt ensures reliable modality input, while the mask and box prompts enable precise salient object localization. Extensive experiments on three public datasets demonstrate that our method achieves state-of-the-art performance. Notably, HyPSAM has remarkable versatility, seamlessly integrating with different RGB-T SOD methods to achieve significant performance gains, thereby highlighting the potential of prompt engineering in this field. The code and results of our method are available at: https://github.com/milotic233/HyPSAM.