Leveraging Pretrained Image Classifiers for Language-Based Segmentation
Preprint
Abstract
Current semantic segmentation models cannot easily generalize to new object classes unseen during train time: they require additional annotated images and retraining. We propose a novel segmentation model that injects visual priors into semantic segmentation architectures, allowing them to segment out new target labels without retraining. As visual priors, we use the activations of pretrained image classifiers, which provide noisy indications of the spatial location of both the target object and distractor objects in the scene. We leverage language semantics to obtain these activations for a target label unseen by the classifier. Further experiments show that the visual priors obtained via language semantics for both relevant and distracting objects are key to our performance.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- SG-One: Similarity Guidance Network for One-Shot Semantic SegmentationIEEE Transactions on Cybernetics, 2020
- Decoupled Spatial Neural Attention for Weakly Supervised Semantic SegmentationIEEE Transactions on Multimedia, 2019
- Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations2019
- Learning to Exploit the Prior Network Knowledge for Weakly Supervised Semantic SegmentationIEEE Transactions on Image Processing, 2019
- Semantic Understanding of Scenes Through the ADE20K DatasetInternational Journal of Computer Vision, 2018
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic SegmentationInternational Journal of Computer Vision, 2014
- Integrating tracking with fine object segmentationImage and Vision Computing, 2013
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009