Semantic Amodal Segmentation
Authors:
Preprint
Abstract
Common visual recognition tasks such as classification, object detection, and semantic segmentation are rapidly reaching maturity, and given the recent rate of progress, it is not unreasonable to conjecture that techniques for many of these problems will approach human levels of performance in the next few years. In this paper we look to the future: what is the next frontier in visual recognition? We offer one possible answer to this question. We propose a detailed image annotation that captures information beyond the visible pixels and requires complex reasoning about full scene structure. Specifically, we create an amodal segmentation of each image: the full extent of each region is marked, not just the visible pixels. Annotators outline and name all salient regions in the image and specify a partial depth order. The result is a rich scene structure, including visible and occluded portions of each region, figure-ground edge information, semantic labels, and object overlap. To date, we have labeled 500 images in the BSDS dataset with at least five annotators per image. Critically, the resulting full scene annotation is surprisingly consistent between annotators. For example, for edge detection our annotations have substantially higher human consistency than the original BSDS edges while providing a greater challenge for existing algorithms. We are currently annotating ~5000 images from the MS COCO dataset.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Nonparametric Scene Parsing via Label Transfer2016
- Microsoft COCO: Common Objects in Context2014
- Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images2013
- A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization.Psychological Bulletin, 2012
- SUN database: Large-scale scene recognition from abbey to zoo2010
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- LabelMe: A Database and Web-Based Tool for Image AnnotationInternational Journal of Computer Vision, 2007
- Supervised Learning of Edges and Object Boundaries2006
- TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation2006
- Finding pictures of objects in large collections of images1996