PhraseCut: Language-Based Image Segmentation in the Wild
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10213-10222
Abstract
We consider the problem of segmenting image regions given a natural language phrase, and study it on a novel dataset of 77,262 images and 345,486 phrase-region pairs. Our dataset is collected on top of the Visual Genome dataset and uses the existing annotations to generate a challenging set of referring phrases for which the corresponding regions are manually annotated. Phrases in our dataset correspond to multiple regions and describe a large number of object and stuff categories as well as their attributes such as color, shape, parts, and relationships with other entities in the image. Our experiments show that the scale and diversity of concepts in our dataset poses significant challenges to the existing state-of-the-art. We systematically handle the long-tail nature of these concepts and present a modular approach to combine category, attribute, and relationship cues that outperforms existing approaches.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Deep Learning and Convolutional Neural Networks for Medical Image Computing2017
- Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence ModelsInternational Journal of Computer Vision, 2016
- Segmentation from Natural Language Expressions2016
- Learning Deep Structure-Preserving Image-Text Embeddings2016
- Deep Residual Learning for Image Recognition2016
- Natural Language Object Retrieval2016
- Generation and Comprehension of Unambiguous Object Descriptions2016
- Neural Module Networks2016
- ReferItGame: Referring to Objects in Photographs of Natural Scenes2014
- Long Short-Term MemoryNeural Computation, 1997