ADPL: Adaptive Dual Path Learning for Domain Adaptation of Semantic Segmentation
- 23 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (8) , 9339-9356
Abstract
To alleviate the need for large-scale pixel-wise annotations, domain adaptation for semantic segmentation trains segmentation models on synthetic data (source) with computer-generated annotations, which can be then generalized to segment realistic images (target). Recently, self-supervised learning (SSL) with a combination of image-to-image translation shows great effectiveness in adaptive segmentation. The most common practice is to perform SSL along with image translation to well align a single domain (source or target). However, in this single-domain paradigm, unavoidable visual inconsistency raised by image translation may affect subsequent learning. In addition, pseudo labels generated by a single segmentation model aligned in either the source or target domain may be not accurate enough for SSL. In this paper, based on the observation that domain adaptation frameworks performed in the source and target domain are almost complementary, we propose a novel adaptive dual path learning (ADPL) framework to alleviate visual inconsistency and promote pseudo-labeling by introducing two interactive single-domain adaptation paths aligned in source and target domain respectively. To fully explore the potential of this dual-path design, novel technologies such as dual path image translation (DPIT), dual path adaptive segmentation (DPAS), dual path pseudo label generation (DPPLG) and Adaptive ClassMix are proposed. The inference of ADPL is extremely simple, only one segmentation model in the target domain is employed. Our ADPL outperforms the state-of-the-art methods by large margins on GTA5 $\rightarrow$ Cityscapes, SYNTHIA $\rightarrow$ Cityscapes and GTA5 $\rightarrow$ BDD100K scenarios. Code and models are available at https://github.com/royee182/DPL .
Keywords
Funding Information
- National Natural Science Foundation of China (62072112)
- Scientific and Technological Innovation Action Plan of Shanghai Science and Technology Committee (22511102202)
This publication has 54 references indexed in Scilit:
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Pyramid Scene Parsing Network2017
- Adversarial Discriminative Domain Adaptation2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Playing for Data: Ground Truth from Computer Games2016
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Fully convolutional networks for semantic segmentation2015
- Visual Domain Adaptation: A survey of recent advancesIEEE Signal Processing Magazine, 2015