Heterogeneous Feature Fusion Module Based on CNN and Transformer for Multiview Stereo Reconstruction
Open Access
- 26 December 2022
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 11 (1) , 112
Abstract
For decades, a vital area of computer vision research has been multiview stereo (MVS), which creates 3D models of a scene using photographs. This study presents an effective MVS network for 3D reconstruction utilizing multiview pictures. Alternative learning-based reconstruction techniques work well, because CNNs (convolutional neural network) can extract only the image’s local features; however, they contain many artifacts. Herein, a transformer and CNN are used to extract the global and local features of the image, respectively. Additionally, hierarchical aggregation and heterogeneous interaction modules were used to improve these features. They are based on the transformer and can extract dense features with 3D consistency and global context that are necessary to provide accurate matching for MVS.Keywords
Funding Information
- National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2022R1A2C200686411)
This publication has 29 references indexed in Scilit:
- Pixelwise View Selection for Unstructured Multi-View Stereo2016
- Structure-from-Motion Revisited2016
- Massively Parallel Multiview Stereopsis by Surface Normal Diffusion2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Image-based 3D scene reconstruction and exploration in augmented realityAutomation in Construction, 2013
- Screened poisson surface reconstructionACM Transactions on Graphics, 2013
- Accurate, Dense, and Robust Multiview StereopsisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- PatchMatchACM Transactions on Graphics, 2009
- Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo2008
- Stereo Processing by Semiglobal Matching and Mutual InformationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007