ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes
Authors:
- Yuhua ChenYCYuhua Chen
- Wen LiWLWen Li
- Luc Van GoolLGLuc Van Gool
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7892-7901
Abstract
Exploiting synthetic data to learn deep models has attracted increasing attention in recent years. However, the intrinsic domain difference between synthetic and real images usually causes a significant performance drop when applying the learned model to real world scenarios. This is mainly due to two reasons: 1) the model overfits to synthetic images, making the convolutional filters incompetent to extract informative representation for real images; 2) there is a distribution difference between synthetic and real data, which is also known as the domain adaptation problem. To this end, we propose a new reality oriented adaptation approach for urban scene semantic segmentation by learning from synthetic data. First, we propose a target guided distillation approach to learn the real image style, which is achieved by training the segmentation model to imitate a pretrained real style model using real images. Second, we further take advantage of the intrinsic spatial structure presented in urban scene images, and propose a spatial-aware adaptation scheme to effectively align the distribution of two domains. These two modules can be readily integrated with existing state-of-the-art semantic segmentation networks to improve their generalizability when adapting from synthetic to real urban scenes. We evaluate the proposed method on Cityscapes dataset by adapting from GTAV and SYNTHIA datasets, where the results demonstrate the effectiveness of our method.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Scale-Aware Alignment of Hierarchical Image Segmentation2016
- Hierarchical Adaptive Structural SVM for Domain AdaptationInternational Journal of Computer Vision, 2016
- Fully convolutional networks for semantic segmentation2015
- Virtual and Real World Adaptation for Pedestrian DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Domain adaptation for object recognition: An unsupervised approach2011
- What you saw is not what you get: Domain adaptation using asymmetric kernel transforms2011
- Putting Objects in PerspectiveInternational Journal of Computer Vision, 2008
- Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories2006