JYJiong Yin

Jiong Yin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jiong Yin has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Hybrid Debiasing Transformer With Adaptive Regularization for Video Moment Localization

    by Jiong Yin, Liang Li, Jiehua Zhang, Chenggang Yan, Hongkui Wang, Yaoqi Sun, Zunjie Zhu

    Video Moment Localization (VML) is a task that seeks to pinpoint the most pertinent segment within an untrimmed video using a linguistic query. Previous works expose the severe data bias issues in VML and note that models avoid understanding visual-textual content by adapting the timestamp distribution. The work investigates data biases from both intrinsic and extrinsic perspectives: The former arises primarily from moment boundary ambiguity and the inputoutput information imbalance. The latter is attributed to the longtail distribution and the semantic bias resulting from the limited tail samples. To reduce the issues, we develop a hybrid multimodal debiasing network with a temporal consistency constraint for VML. Firstly, we propose a multi-temporal Transformer to alleviate boundary ambiguity by merging frame-wise features into segment-wise representations and dynamically aligning with moment boundaries. Subsequently, we implement a temporal consistency constraint to accentuate action information from complex moment context and mitigate the intrinsic bias caused by information imbalance. Moreover, we develop a hybrid linguistic activation module to mitigate the long-tail bias, which offers prior guidance to emphasize distinguishing clues from tail samples. Additionally, we introduce the prior-guided Transformer to alleviate the semantic bias by learning the global semantics of sentences, thereby circumventing the tail-sample overfitting issue. Comprehensive experiments demonstrate the efficacy of our proposed method across three datasets.


  • Generalizing Beyond Patterns: Dynamic Moment Query Recalibrating for Out-of-Distribution Video Temporal Localization

    by Zhuo Tao, Liang Li, Yunbin Tu, Jiong Yin, Beichen Zhang, Zheng-Jun Zha, Qingming Huang

    Video temporal localization (VTL) aims to accurately identify the temporal boundary of a specific segment corresponding to a sentence query from an untrimmed video. Recently, DETR-based methods have emerged as effective solutions to VTL. The primary idea of these methods is to fuse multi-modal features and refine moment queries to obtain precise locations. Despite their success, these methods adopt fixed trainable moment queries, which implicitly model the statistical patterns of groundtruth segments in the training set and thus limit the model's generalization ability to out-of-distribution scenarios. To tackle this problem, we propose a Dynamic moment Query Recalibrated DEtection TRansformer (DQR-DETR) that adaptively generates and recalibrates multi-scale moment queries on the target segments for out-of-distribution VTL. Our method contains three key parts: (1) progressive semantic regularization that performs primitive modality consensus loss and refined semantic calibration loss at two phases for better global and fine-grained cross-modal associations; (2) hierarchical query decoupling that first dynamically extracts multi-scale and context-aware event queries from the joint visual-language representations to cover a wide range of potential events in video, and then select the sentence-aware queries with semantic guidance; (3) moment query recalibrating that performs coarse-to-fine modulation on multi-scale queries by first activating relevant queries with video highlight and sentence semantic guidance, and then refining them to focus on the target segments. Extensive experiments on QVHighlights, Charades-STA and TACoS datasets show that our DQR-DETR achieves state-of-the-art performance, especially in out-of-distribution scenarios where our method substantially outperforms existing approaches by large margins.

  • Open Access

    Reducing Intrinsic and Extrinsic Data Biases for Moment Localization with Natural Language

    by Jiong Yin, Liang Li, Jiehua Zhang, Chenggang Yan, Lei Zhang, Zunjie Zhu

    Moment Localization with Natural Language (MLNL) aims to locate the target moment from an untrimmed video by a linguistic query. Recent works reveal the severe data bias problem in MLNL and point out that the multi-modal content may not be understood by fitting the timestamp distribution. In this paper, we study the data biases on the intrinsic and extrinsic aspects: the former is mainly caused by the ambiguity of the moment boundary and the information imbalance between input and output; The latter results from the long-tail distribution of moments in MLNL datasets. To alleviate this, we propose a hybrid multi-modal debiasing network with temporal consistency constraint for MLNL. Specifically, we first design the multi-temporal Transformer to mitigate the ambiguity of boundary by integrating frame-wise features into segment-wise and dynamically matching with moment boundaries. Then, we introduce the temporal consistency constraint that highlights the action information in complex moment content to overcome the intrinsic bias from information imbalance.Furthermore, we design the hybrid linguistic activating module with external knowledge to relieve the extrinsic bias, which introduces a prior guidance to focus the discriminative information from the tail samples. Extensive experiments on three public datasets demonstrate that our model outperforms the existing methods.