Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the Wild
Authors:
- 12 January 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149) , 1866-1881
Abstract
Semantic segmentation, unifying most navigational perception tasks at the pixel level has catalyzed striking progress in the field of autonomous transportation. Modern Convolution Neural Networks (CNNs) are able to perform semantic segmentation both efficiently and accurately, particularly owing to their exploitation of wide context information. However, most segmentation CNNs are benchmarked against pinhole images with limited Field of View (FoV). Despite the growing popularity of panoramic cameras to sense the surroundings, semantic segmenters have not been comprehensively evaluated on omnidirectional wide-FoV data, which features rich and distinct contextual information. In this paper, we propose a concurrent horizontal and vertical attention module to leverage width-wise and height-wise contextual priors markedly available in the panoramas. To yield semantic segmenters suitable for wide-FoV images, we present a multi-source omni-supervised learning scheme with panoramic domain covered in the training via data distillation. To facilitate the evaluation of contemporary CNNs in panoramic imagery, we put forward the Wild PAnoramic Semantic Segmentation (WildPASS) dataset, comprising images from all around the globe, as well as adverse and unconstrained scenes, which further reflects perception challenges of navigation applications in the real world. A comprehensive variety of experiments demonstrates that the proposed methods enable our high-efficiency architecture to attain significant accuracy gains, outperforming the state of the art in panoramic imagery domains.Keywords
Funding Information
- Federal Ministry of Labor and Social Affairs (BMAS) through the AccessibleMaps project (01KM151112)
- University of Excellence through the “KIT Future Fields”
- KIT-Publication Fund of the Karlsruhe Institute of Technology
- Hangzhou SurImage Technology Company Ltd.
- Hangzhou KrVision Technology Company Ltd.
This publication has 49 references indexed in Scilit:
- ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems, 2017
- CNN based semantic segmentation for urban traffic scenes using fisheye camera2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Fully convolutional networks for semantic segmentation2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- A Fast Image Stitching Algorithm via Multiple-Constraint Corner MatchingMathematical Problems in Engineering, 2013
- Visual Place Recognition with Repetitive Structures2013