Context-Aware Mixup for Domain Adaptive Semantic Segmentation
- 14 September 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (2) , 804-817
Abstract
Unsupervised domain adaptation (UDA) aims to adapt a model of the labeled source domain to an unlabeled target domain. Existing UDA-based semantic segmentation approaches always reduce the domain shifts in pixel level, feature level, and output level. However, almost all of them largely neglect the contextual dependency, which is generally shared across different domains, leading to less-desired performance. In this paper, we propose a novel Context-Aware Mixup (CAMix) framework for domain adaptive semantic segmentation, which exploits this important clue of context-dependency as explicit prior knowledge in a fully end-to-end trainable manner for enhancing the adaptability toward the target domain. Firstly, we present a contextual mask generation strategy by leveraging the accumulated spatial distributions and prior contextual relationships. The generated contextual mask is critical in this work and will guide the context-aware domain mixup on three different levels. Besides, provided the context knowledge, we introduce a significance-reweighted consistency loss to penalize the inconsistency between the mixed student prediction and the mixed teacher prediction, which alleviates the negative transfer of the adaptation, e.g ., early performance degradation. Extensive experiments and analysis demonstrate the effectiveness of our method against the state-of-the-art approaches on widely-used UDA benchmarks.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2019YFC1521104)
- National Natural Science Foundation of China (72192821)
- National Natural Science Foundation of China (61972157)
- Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102)
- Shanghai Science and Technology Commission (21511101200)
- Shanghai Science and Technology Commission (22YF1420300)
- Art major project of National Social Science Fund (I8ZD22)
This publication has 81 references indexed in Scilit:
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Playing for Data: Ground Truth from Computer Games2016
- Computation and Memory Efficient Image SegmentationIEEE Transactions on Circuits and Systems for Video Technology, 2016
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Fully convolutional networks for semantic segmentation2015
- Microsoft COCO: Common Objects in Context2014
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998