Progressive Frame-Proposal Mining for Weakly Supervised Video Object Detection
- 15 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 33 (10577149) , 1560-1573
Abstract
In this paper, we focus on the weakly supervised video object detection problem, where each training video is only tagged with object labels, without any bounding box annotations of objects. To effectively train object detectors from such weakly-annotated videos, we propose a Progressive Frame-Proposal Mining (PFPM) framework by exploiting discriminative proposals in a coarse-to-fine manner. First, we design a flexible Multi-Level Selection (MLS) scheme, with explicit guidance of video tags. By selecting object-relevant frames and mining important proposals from these frames, the proposed MLS can effectively reduce frame redundancy as well as improve proposal effectiveness to boost weakly-supervised detectors. Moreover, we develop a novel Holistic-View Refinement (HVR) scheme, which can globally evaluate importance of proposals among frames, and thus correctly refine pseudo ground truth boxes for training video detectors in a self-supervised manner. Finally, we evaluate the proposed PFPM on a large-scale benchmark for video object detection, on ImageNet VID, under the setting of weak annotations. The experimental results demonstrate that our PFPM significantly outperforms the state-of-the-art weakly-supervised detectors.Keywords
Funding Information
- Australian Research Council under Discovery Program (DP240100181)
- National Key Research and Development Program of China (2022ZD0160505)
- National Natural Science Foundation of China (62272450)
- Joint Laboratory of CAS-HK
- Data61 Ph.D. Scholarships
This publication has 56 references indexed in Scilit:
- Mining Inter-Video Proposal Relations for Video Object Detection2020
- Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos2020
- Video Object Detection with an Aligned Spatial-Temporal Memory2018
- Weakly Supervised Region Proposal Network and Object Detection2018
- Object Detection in Video with Spatiotemporal Sampling Networks2018
- ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization2016
- Semantic object segmentation via detection in weakly labeled video2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Edge Boxes: Locating Object Proposals from Edges2014
- Selective Search for Object RecognitionInternational Journal of Computer Vision, 2013