HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images
Open Access
- 11 January 2024
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 16 (2) , 295
Abstract
Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, with monocular height estimation offering wide-ranging data sources and convenient deployment. This paper addresses the existing challenges in monocular height estimation methods, namely the difficulty in simultaneously achieving high-quality instance-level height and edge reconstruction, along with high computational complexity. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification–regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification–regression Height Generator (ICG). MIB supplements the fixed sample grid in the CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification–regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness. Finally, the study conducts experimental validations on the Vaihingen and Potsdam datasets, with results demonstrating that our proposed method surpasses existing techniques.Keywords
All Related Versions
Funding Information
- Key Laboratory Fund of Chinese Academy Of Sciences (CXJJ-23S032, CXJJ-22S032)
- the Key Laboratory fund of Chinese Academy of Sciences (CXJJ-23S032, CXJJ-22S032)
This publication has 44 references indexed in Scilit:
- A Critical Review of High and Very High-Resolution Remote Sensing Approaches for Detecting and Mapping Slums: Trends, Challenges and Emerging OpportunitiesUrban Science, 2018
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction2017
- Effective Approaches to Attention-based Neural Machine Translation2015
- Discrete-Continuous Depth Estimation from a Single Image2014
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Monocular Visual Scene Understanding: Understanding Multi-Object Traffic ScenesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- HIGH-PRECISION DEM PRODUCTION FOR SPACEBORNE STEREO SAR IMAGES BASED ON SIFT MATCHING AND REGION-BASED LEAST SQUARES MATCHINGThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012
- Learning the right model: Efficient max-margin learning in Laplacian CRFs2012
- Very High-Resolution Remote Sensing: Challenges and Opportunities [Point of View]Proceedings of the IEEE, 2012
- Analysis of the factors affecting LiDAR DTM accuracy in a steep shrub areaInternational Journal of Digital Earth, 2010