Federated Hallucination Translation and Source-Free Regularization Adaptation in Decentralized Domain Adaptation for Foggy Scene Understanding
- 25 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1601-1616
Abstract
Semantic foggy scene understanding (SFSU) emerges a challenging task under out-of-domain distribution (OD) due to uncertain cognition caused by degraded visibility. With the strong assumption of data centralization, unsupervised domain adaptation (UDA) reduces vulnerability under OD scenario. Whereas, enlarged domain gap and growing privacy concern heavily challenge conventional UDA. Motivated by gap decomposition and data decentralization, we establish a decentralized domain adaptation (DDA) framework called Translate thEn Adapt (abbr. TEA) for privacy preservation. Our highlights lie in: (1) Regarding federated hallucination translation, a Disentanglement and Contrastive-learning based Generative Adversarial Network (abbr. DisCoGAN) is proposed to impose contrastive prior and disentangle latent space in cycle-consistent translation. To yield domain hallucination, client minimizes cross-entropy of local classifier but maximizes entropy of global model to train translator. (2) Regarding source-free regularization adaptation, a Prototypical-knowledge based Regularization Adaptation (abbr. ProRA) is presented to align joint distribution in output space. Soft adversarial learning relaxes binary label to rectify inter-domain discrepancy and inner-domain divergence. Structure clustering and entropy minimization drive intra-class features closer and inter-class features apart. Extensive experiments exhibit efficacy of our TEA which achieves 55.26% or 46.25% mIoU in adaptation from GTA5 to Foggy Cityscapes or Foggy Zurich, outperforming other DDA methods for SFSU. Code will be released at URL.Keywords
Funding Information
- National Natural Science Foundation of China (62372408, 62027810)
- National Key Research and Development Program of China (2021YFB2701100, 2019YFF0302601)
- Zhejiang Provincial Key Research and Development Program of China (2021C01105)
- Natural Science Foundation of Ningbo Municipality (2022J183)
- Key Program of the National Natural Science Foundation of China (62433010)
- Major Research Plan of the National Natural Science Foundation of China (92148204)
- Hunan Leading Talent of Technological Innovation (2022RC3063)
- Hunan Science Fund for Distinguished Young Scholars (2021JJ10025)
- Joint Open Foundation of State Key Laboratory of Robotics (2021-KF-22-17)
This publication has 58 references indexed in Scilit:
- Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation2022
- Classes Matter: A Fine-Grained Adversarial Approach to Cross-Domain Semantic Segmentation2020
- Contrastive Learning for Unpaired Image-to-Image Translation2020
- DRIT++: Diverse Image-to-Image Translation via Disentangled RepresentationsInternational Journal of Computer Vision, 2020
- Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene UnderstandingInternational Journal of Computer Vision, 2019
- Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training2018
- Multimodal Unsupervised Image-to-Image Translation2018
- Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding2018
- Semantic Foggy Scene Understanding with Synthetic DataInternational Journal of Computer Vision, 2018
- Playing for Data: Ground Truth from Computer Games2016