RRT-MVS: Recurrent Regularization Transformer for Multi-View Stereo
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (4) , 3994-4002
Abstract
Learning-based multi-view stereo methods aim to predict depth maps for reconstructing dense point clouds. These methods rely on regularization to reduce redundancy in the cost volume. However, existing methods have limitations: CNN-based regularization is restricted to local receptive fields, while Transformer-based regularization struggles with handling depth discontinuities. These limitations often result in inaccurate depth maps with significant noise, particularly noticeable in the boundary and background regions. In this paper, we propose a Recurrent Regularization Transformer for Multi-View Stereo (RRT-MVS), which addresses these limitations by regularizing the cost volume separately for depth and spatial dimensions. Specifically, we introduce Recurrent Self-Attention (R-SA) to aggregate global matching costs within and across the cost maps and filter out noisy feature correlations. Additionally, we present Depth Residual Attention (DRA) to aggregate depth correlations within the cost volume and a Positional Adapter (PA) to enhance 3D positional awareness in each 2D cost map, further augmenting the effectiveness of R-SA. Experimental results demonstrate that RRT-MVS achieves state-of-the-art performance on the DTU and Tanks-and-Temples datasets. Notably, RRT-MVS ranks first on both the Tanks-and-Temples intermediate and advanced benchmarks among all published methods.Keywords
This publication has 5 references indexed in Scilit:
- Vis-MVSNet: Visibility-Aware Multi-view Stereo NetworkInternational Journal of Computer Vision, 2022
- Multi-Scale Geometric Consistency Guided and Planar Prior Assisted Multi-View StereoIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- Learning Inverse Depth Regression for Pixelwise Visibility-Aware Multi-View Stereo NetworksInternational Journal of Computer Vision, 2022
- Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking2020
- Large-Scale Data for Multiple-View StereopsisInternational Journal of Computer Vision, 2016