Jian Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Jian Wang has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Wang Lin
Zhejiang University, Hangzhou, China
2 shared publications - Haibin Cai
East China Normal University
1 shared publication Hao Jiang
1 shared publication- Hao Jiang
Alibaba Group, Hangzhou, China
1 shared publication Jingyuan Chen
1 shared publication- Linjun Li
Meituan, Shanghai, China
1 shared publication
Latest Publications
- 4 June 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 290
Recognize-and-tell: Generating video captions with textual cue in scene
- 4 June 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 290
Recognize-and-tell: Generating video captions with textual cue in scene
- 8 June 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12707, 200-208
Query graph attention for video relation detection
As a bridge to connect vision and language, visual relations between objects, visual relation provide a more comprehensive visual content understanding beyond objects. Most previous works adopt the track-to-detect framework for video visual relation detection (VidVRD), which cannot capture long-term spatio- temporal contexts in different stages and also suffers from inefficiency. In this work, we propose a query-based method for video visual relation detection. Our model exploits graph structure to autoregressively generate relation graphs with spatio-temporal contexts and uses an attentional graph convolutional network to fuse the contexts. Experiments on benchmark datasets ImageNet-VidVRD demonstrate the accuracy of our method.
- 8 June 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12707, 200-208
Query graph attention for video relation detection
As a bridge to connect vision and language, visual relations between objects, visual relation provide a more comprehensive visual content understanding beyond objects. Most previous works adopt the track-to-detect framework for video visual relation detection (VidVRD), which cannot capture long-term spatio- temporal contexts in different stages and also suffers from inefficiency. In this work, we propose a query-based method for video visual relation detection. Our model exploits graph structure to autoregressively generate relation graphs with spatio-temporal contexts and uses an attentional graph convolutional network to fuse the contexts. Experiments on benchmark datasets ImageNet-VidVRD demonstrate the accuracy of our method.
- 5 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (8) , 7080-7094
Multi-Granularity Relational Attention Network for Audio-Visual Question Answering
Recent methods for video question answering (VideoQA), aiming to generate answers based on given questions and video content, have made significant progress in cross-modal interaction. From the perspective of video understating, these existing frameworks concentrate on the various levels of visual content, partially assisted by subtitles. However, audio information is also instrumental in helping get correct answers, especially in videos with real-life scenarios. Indeed, in some cases, both audio and visual contents are required and complement each other to answer questions, which is defined as audio-visual question answering (AVQA). In this paper, we focus on importing raw audio for AVQA and contribute in three ways. Firstly, due to no dataset annotating QA pairs for raw audio, we introduce E-AVQA, a manually annotated and large-scale dataset involving multiple modalities. E-AVQA consists of 34,033 QA pairs on 33,340 clips of 18,786 videos from the e-commerce scenarios. Secondly, we propose a multi-granularity relational attention method with contrastive constraints between audio and visual features after the interaction, named MGN, which captures local sequential representation by leveraging the pairwise potential attention mechanism and obtains global multi-modal representation via designing the novel ternary potential attention mechanism. Thirdly, our proposed MGN outperforms the baseline on dataset E-AVQA, achieving 20.73% on WUPS@0.0 and 19.81% on BLEU@1, demonstrating its superiority with at least 1.02 improvement on WUPS@0.0 and about 10% on timing complexity over the baseline.
- 5 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (8) , 7080-7094
Multi-Granularity Relational Attention Network for Audio-Visual Question Answering
Recent methods for video question answering (VideoQA), aiming to generate answers based on given questions and video content, have made significant progress in cross-modal interaction. From the perspective of video understating, these existing frameworks concentrate on the various levels of visual content, partially assisted by subtitles. However, audio information is also instrumental in helping get correct answers, especially in videos with real-life scenarios. Indeed, in some cases, both audio and visual contents are required and complement each other to answer questions, which is defined as audio-visual question answering (AVQA). In this paper, we focus on importing raw audio for AVQA and contribute in three ways. Firstly, due to no dataset annotating QA pairs for raw audio, we introduce E-AVQA, a manually annotated and large-scale dataset involving multiple modalities. E-AVQA consists of 34,033 QA pairs on 33,340 clips of 18,786 videos from the e-commerce scenarios. Secondly, we propose a multi-granularity relational attention method with contrastive constraints between audio and visual features after the interaction, named MGN, which captures local sequential representation by leveraging the pairwise potential attention mechanism and obtains global multi-modal representation via designing the novel ternary potential attention mechanism. Thirdly, our proposed MGN outperforms the baseline on dataset E-AVQA, achieving 20.73% on WUPS@0.0 and 19.81% on BLEU@1, demonstrating its superiority with at least 1.02 improvement on WUPS@0.0 and about 10% on timing complexity over the baseline.