Semantic Labeling of High-Resolution Images Using EfficientUNets and Transformers
Open Access
- 18 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-13
Abstract
Semantic segmentation necessitates approaches that learn high-level characteristics while dealing with enormous quantities of data. Convolutional neural networks (CNNs) can learn unique and adaptive features to achieve this aim. However, due to the large size and high spatial resolution of remote sensing images, these networks cannot efficiently analyze an entire scene. Recently, deep transformers have proven their capability to record global interactions between different objects in the image. In this paper, we propose a new segmentation model that combines convolutional neural networks with transformers, and show that this mixture of local and global feature extraction techniques provides significant advantages in remote sensing segmentation. In addition, the proposed model includes two fusion layers that are designed to efficiently represent multimodal inputs and output of the network. The input fusion layer extracts feature maps summarizing the relationship between image content and elevation maps (DSM). The output fusion layer uses a novel multitask segmentation strategy where class labels are identified using class-specific feature extraction layers and loss functions. Finally, a fast-marching method is used to convert unidentified class labels to their closest known neighbors. Our results demonstrate that the proposed method improves segmentation accuracy compared to state-of-the-art techniques.Keywords
All Related Versions
Funding Information
- Information and Communication Technology Fund (ICT), Telecommunications Regulatory Authority (TRA), Abu Dhabi, United Arab Emirates
This publication has 61 references indexed in Scilit:
- Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution ImagesRemote Sensing, 2017
- Object-Based Convolutional Neural Network for High-Resolution Imagery ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Fully convolutional networks for building and road extraction: Preliminary results2016
- Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural NetworksIEEE Transactions on Geoscience and Remote Sensing, 2016
- Effective semantic pixel labelling with convolutional networks and Conditional Random Fields2015
- Fully convolutional networks for semantic segmentation2015
- A Non-Local Algorithm for Image Denoising2005
- Random forest classifier for remote sensing classificationInternational Journal of Remote Sensing, 2005
- An Image Inpainting Technique Based on the Fast Marching MethodJournal of Graphics Tools, 2004