Both Style and Distortion Matter: Dual-Path Unsupervised Domain Adaptation for Panoramic Semantic Segmentation
Preprint
Abstract
The ability of scene understanding has sparked active research for panoramic image semantic segmentation. However, the performance is hampered by distortion of the equirectangular projection (ERP) and a lack of pixel-wise annotations. For this reason, some works treat the ERP and pinhole images equally and transfer knowledge from the pinhole to ERP images via unsupervised domain adaptation (UDA). However, they fail to handle the domain gaps caused by: 1) the inherent differences between camera sensors and captured scenes; 2) the distinct image formats (e.g., ERP and pinhole images). In this paper, we propose a novel yet flexible dual-path UDA framework, DPPASS, taking ERP and tangent projection (TP) images as inputs. To reduce the domain gaps, we propose cross-projection and intra-projection training. The cross-projection training includes tangent-wise feature contrastive training and prediction consistency training. That is, the former formulates the features with the same projection locations as positive examples and vice versa, for the models' awareness of distortion, while the latter ensures the consistency of cross-model predictions between the ERP and TP. Moreover, adversarial intra-projection training is proposed to reduce the inherent gap, between the features of the pinhole images and those of the ERP and TP images, respectively. Importantly, the TP path can be freely removed after training, leading to no additional inference cost. Extensive experiments on two benchmarks show that our DPPASS achieves +1.06$\%$ mIoU increment than the state-of-the-art approaches.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Transfer Beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain AdaptationIEEE Transactions on Intelligent Transportation Systems, 2021
- Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the WildIEEE Transactions on Image Processing, 2021
- Real-Time Semantic Segmentation With Fast AttentionIEEE Robotics and Automation Letters, 2020
- Omnisupervised Omnidirectional Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems, 2020
- Restricted Deformable Convolution-Based Road Scene Semantic Segmentation Using Surround View CamerasIEEE Transactions on Intelligent Transportation Systems, 2019
- PASS: Panoramic Annular Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems, 2019
- Distortion-aware CNNs for Spherical Images2018
- Automatic Parking Based on a Bird's Eye View Vision SystemAdvances in Mechanical Engineering, 2014