HCNet: Hierarchical Context Network for Semantic Segmentation
Open Access
- 1 October 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 179213-179223
Abstract
Global context information is vital in visual understanding problems, especially in pixel-level semantic segmentation. The mainstream methods adopt the self-attention mechanism to model global context information. However, pixels belonging to different classes usually have weak feature correlation. Modeling the global pixel-level correlation matrix indiscriminately is extremely redundant in the self-attention mechanism. In order to solve the above problem, we propose a hierarchical context network to differentially model homogeneous pixels with strong correlations and heterogeneous pixels with weak correlations. Specifically, we first propose a multi-scale guided pre-segmentation module to divide the entire feature map into different classed-based homogeneous regions. Within each homogeneous region, we design the pixel context module to capture pixel-level correlations. Subsequently, different from the self-attention mechanism that still models weak heterogeneous correlations in a dense pixel-level manner, the region context module is proposed to model sparse region-level dependencies using a unified representation of each region. Through aggregating fine-grained pixel context features and coarse-grained region context features, our proposed network can not only hierarchically model global context information but also harvest multi-granularity representations to more robustly identify multi-scale objects. We evaluate our approach on Cityscapes and the ISPRS Vaihingen dataset. Without Bells or Whistles, our approach realizes a mean IoU of 82.8% and overall accuracy of 91.4% on Cityscapes and ISPRS Vaihingen test set, achieving state-of-the-art results.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62072345, 41671382)
- State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing
This publication has 39 references indexed in Scilit:
- Scale-Invariant Feature Transform (SIFT)2016
- Hierarchical Attention Networks for Document Classification2016
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Effective semantic pixel labelling with convolutional networks and Conditional Random Fields2015
- Going deeper with convolutions2015
- Fully convolutional networks for semantic segmentation2015
- A brief study of image segmentation using Thresholding Technique on a Noisy Image2014
- Histograms of Oriented Gradients for Human Detection2005
- A color clustering technique for image segmentationComputer Vision, Graphics, and Image Processing, 1990