Not All Areas Are Equal: Transfer Learning for Semantic Segmentation via Hierarchical Region Selection
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4355-4364
Abstract
The success of deep neural networks for semantic segmentation heavily relies on large-scale and well-labeled datasets, which are hard to collect in practice. Synthetic data offers an alternative to obtain ground-truth labels for free. However, models directly trained on synthetic data often struggle to generalize to real images. In this paper, we consider transfer learning for semantic segmentation that aims to mitigate the gap between abundant synthetic data (source domain) and limited real data (target domain). Unlike previous approaches that either learn mappings to target domain or finetune on target images, our proposed method jointly learn from real images and selectively from realistic pixels in synthetic images to adapt to the target domain. Our key idea is to have weighting networks to score how similar the synthetic pixels are to real ones, and learn such weighting at pixel-, region- and image-levels. We jointly learn these hierarchical weighting networks and segmentation network in an end-to-end manner. Extensive experiments demonstrate that our proposed approach significantly outperforms other existing baselines, and is applicable to scenarios with extremely limited real images.Keywords
This publication has 20 references indexed in Scilit:
- Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks2017
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network2016
- Semantic Image Segmentation via Deep Parsing Network2015
- From generic to specific deep representations for visual recognition2015
- Fully convolutional networks for semantic segmentation2015
- A theory of learning from different domainsMachine Learning, 2009
- Boosting for transfer learning2007