ATMNet: Adaptive Texture Migration Network for Guided Depth Super-Resolution

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Guided depth super-resolution (GDSR) aims to enhance the level of detail in low-resolution depth images by utilizing the information present in the corresponding high-resolution RGB images. While existing methods utilize different approaches to guide the RGB image to the source image, they often ignore the texture similarity between these two images and usually suffer from unsatisfactory outline reconstruction of the depth map. In this article, we introduce an adaptive texture migration network (ATMNet) designed to mine rich feature information from RGB images and migrate them to the depth image. Specifically, we propose a multi-modal feature extractor (MMFE) to extract private and shared features between the depth map and RGB image. In addition, we present a texture migration module (TMM) to remap and fuse the features extracted from the raw image pairs. Last but not least, we develop a weighted adaptive loss to enhance the reconstruction of the edge areas in the depth map. Extensive experiments on public datasets such as Middlebury, NYUv2, and DIML demonstrate that our method outperforms the existing state-of-the-art GDSR methods and strikes a remarkable balance between performance and efficiency. The source code is available at https://github.com/MuggleTan/ATMNet .
Funding Information
  • National Natural Science Foundation of China (62472443)
  • National Key R&D Program of China (2023ZD0508201)
  • Frontier Cross Project of Central South University (2023QYJC008)
  • General Project of Xiangjiang Lab (23XJ03008)
  • Key Research and Development Plan of Hunan Province (2023SK2027)
  • Hunan Provincial Natural Science Foundation (2022JJ30851, 2023JJ70061, 2024JJ3032)

This publication has 30 references indexed in Scilit: