A stereo matching algorithm based on the improved PSMNet
Open Access
- 19 August 2021
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (8) , e0251657
Abstract
Deep learning based on a convolutional neural network (CNN) has been successfully applied to stereo matching. Compared with the traditional method, the speed and accuracy of this method have been greatly improved. However, the existing stereo matching framework based on a CNN often encounters two problems. First, the existing stereo matching network has many parameters, which leads to the matching running time being too long. Second, the disparity estimation is inadequate in some regions where reflections, repeated textures, and fine structures may lead to ill-posed problems. Through the lightweight improvement of the PSMNet (Pyramid Stereo Matching Network) model, the common matching effect of ill-conditioned areas such as repeated texture areas and weak texture areas is solved. In the feature extraction part, ResNeXt is introduced to learn unitary feature extraction, and the ASPP (Atrous Spatial Pyramid Pooling) module is trained to extract multiscale spatial feature information. The feature fusion module is designed to effectively fuse the feature information of different scales to construct the matching cost volume. The improved 3D CNN uses the stacked encoding and decoding structure to further regularize the matching cost volume and obtain the corresponding relationship between feature points under different parallax conditions. Finally, the disparity map is obtained by a regression. We evaluate our method on the Scene Flow, KITTI 2012, and KITTI 2015 stereo datasets. The experiments show that the proposed stereo matching network achieves a comparable prediction accuracy and much faster running speed compared with PSMNet.Keywords
Funding Information
- National Natural Science Foundation of China (51875266)
This publication has 21 references indexed in Scilit:
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentationJournal of Visual Communication and Image Representation, 2017
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Fully Convolutional Networks for Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Object scene flow for autonomous vehicles2015
- Computing the stereo matching cost with a convolutional neural network2015
- Fast dense stereo correspondences by binary locality sensitive hashing2015
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Local Density Encoding for Robust Stereo MatchingIEEE Transactions on Circuits and Systems for Video Technology, 2014
- An object oriented segmentation on analog CNN chipIEEE Transactions on Circuits and Systems I: Regular Papers, 2003