Semantics-Preserving Graph Propagation for Zero-Shot Object Detection
- 31 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 29 (10577149) , 8163-8176
Abstract
Most existing object detection models are restricted to detecting objects from previously seen categories, an approach that tends to become infeasible for rare or novel concepts. Accordingly, in this paper, we explore object detection in the context of zero-shot learning, i.e., Zero-Shot Object Detection (ZSD), to concurrently recognize and localize objects from novel concepts. Existing ZSD algorithms are typically based on a strict mapping-transfer strategy that suffers from a significant visual-semantic gap. To bridge the gap, we propose a novel Semantics-Preserving Graph Propagation model for ZSD based on Graph Convolutional Networks (GCN). More specifically, we develop a graph construction module to flexibly build category graphs by leveraging diverse correlations between category nodes; this is followed by two semantics-preserving graph propagation modules that enhance both category and region representations. Benefiting from the multi-step graph propagation process, both the semantic description and structural knowledge exhibited in prior category graphs can be effectively leveraged to boost the generalization capability of the learned projection function. Experiments on existing seen/unseen splits of three popular object detection datasets demonstrate that the proposed approach performs favorably against state-of-the-art ZSD methods.Keywords
Funding Information
- Program of China Scholarship Council
- National Basic Research Program of China (2016YFB1000903)
- National Nature Science Foundation of China (61872287, 61772411, 61532015)
- Innovative Research Group of the National Natural Science Foundation of China (61721002)
- Innovation Research Team of Ministry of Education (IRT 17R86)
- Project of China Knowledge Center for Engineering Science and Technology
This publication has 28 references indexed in Scilit:
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- Deep Residual Learning for Image Recognition2016
- Weakly Supervised Deep Detection Networks2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Large-Scale Object Classification Using Label Relation Graphs2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Glove: Global Vectors for Word Representation2014
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- The Graph Neural Network ModelIEEE Transactions on Neural Networks, 2008
- A new model for learning in graph domains2006