ESMStereo: Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching
Preprint
Abstract
Stereo matching has become an increasingly important component of modern autonomous systems. Developing deep learning-based stereo matching models that deliver high accuracy while operating in real-time continues to be a major challenge in computer vision. In the domain of cost-volume-based stereo matching, accurate disparity estimation depends heavily on large-scale cost volumes. However, such large volumes store substantial redundant information and also require computationally intensive aggregation units for processing and regression, making real-time performance unattainable. Conversely, small-scale cost volumes followed by lightweight aggregation units provide a promising route for real-time performance, but lack sufficient information to ensure highly accurate disparity estimation. To address this challenge, we propose the Enhanced Shuffle Mixer (ESM) to mitigate information loss associated with small-scale cost volumes. ESM restores critical details by integrating primary features into the disparity upsampling unit. It quickly extracts features from the initial disparity estimation and fuses them with image features. These features are mixed by shuffling and layer splitting then refined through a compact feature-guided hourglass network to recover more detailed scene geometry. The ESM focuses on local contextual connectivity with a large receptive field and low computational cost, leading to the reconstruction of a highly accurate disparity map at real-time. The compact version of ESMStereo achieves an inference speed of 116 FPS on high-end GPUs and 91 FPS on the AGX Orin.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Lightstereo: Channel Boost is All You Need for Efficient 2D Cost Aggregation2025
- DCVSMNet: Double Cost Volume Stereo Matching NetworkNeurocomputing, 2024
- MoCha-Stereo: Motif Channel Attention Network for Stereo Matching2024
- Toward Accurate and Real-Time Binocular Vision: A Lightweight Stereo Matching Network for RGB Stereo ImagesIEEE Sensors Journal, 2023
- Real-time stereo matching with high accuracy via Spatial Attention-Guided UpsamplingApplied Intelligence, 2023
- EBStereo: edge-based loss function for real-time stereo matchingThe Visual Computer, 2023
- Feature back-projection guided residual refinement for real-time stereo matching networkSignal Processing: Image Communication, 2022
- Multi-Dimensional Cooperative Network for Stereo MatchingIEEE Robotics and Automation Letters, 2021
- Feature-Guided Spatial Attention Upsampling for Real-Time Stereo Matching NetworkIEEE MultiMedia, 2020
- EdgeStereo: An Effective Multi-task Learning Network for Stereo Matching and Edge DetectionInternational Journal of Computer Vision, 2020