Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints from Limited Training Data
Authors:
Preprint
Abstract
Detecting semantic parts of an object is a challenging task in computer vision, particularly because it is hard to construct large annotated datasets due to the difficulty of annotating semantic parts. In this paper we present an approach which learns from a small training dataset of annotated semantic parts, where the object is seen from a limited range of viewpoints, but generalizes to detect semantic parts from a much larger range of viewpoints. Our approach is based on a matching algorithm for finding accurate spatial correspondence between two images, which enables semantic parts annotated on one image to be transplanted to another. In particular, this enables images in the training dataset to be matched to a virtual 3D model of the object (for simplicity, we assume that the object viewpoint can be estimated by standard techniques). Then a clustering algorithm is used to annotate the semantic parts of the 3D virtual model. This virtual 3D model can be used to synthesize annotated images from a large range of viewpoint. These can be matched to images in the test set, using the same matching algorithm, to detect semantic parts in novel viewpoints of the object. Our algorithm is very simple, intuitive, and contains very few parameters. We evaluate our approach in the car subclass of the VehicleSemanticPart dataset. We show it outperforms standard deep network approaches and, in particular, performs much better on novel viewpoints. For facilitating the future research, code is available: https://github.com/ytongbai/SemanticPartDetectionKeywords
All Related Versions
This publication has 12 references indexed in Scilit:
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Viewpoint-aware object detection and continuous pose estimationImage and Vision Computing, 2012
- Fast Cost-Volume Filtering for Visual Correspondence and BeyondIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Measuring the Objectness of Image WindowsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Nonparametric Scene Parsing via Label TransferIEEE Transactions on Pattern Analysis and Machine Intelligence, 2011
- SIFT Flow: Dense Correspondence across Scenes and Its ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Object Detection with Discriminatively Trained Part-Based ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Histograms of Oriented Gradients for Human Detection2005
- Robust wide-baseline stereo from maximally stable extremal regionsImage and Vision Computing, 2004
- A multiple-baseline stereoIEEE Transactions on Pattern Analysis and Machine Intelligence, 1993