Global Matching with Overlapping Attention for Optical Flow Estimation
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17571-17580
Abstract
Optical flow estimation is a fundamental task in computer vision. Recent direct-regression methods using deep neural networks achieve remarkable performance improvement. However, they do not explicitly capture long-term motion correspondences and thus cannot handle large motions effectively. In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet. In GMFlowNet, global matching is efficiently calculated by applying argmax on 4D cost volumes. Additionally, to improve the matching quality, we propose patch-based overlapping attention to extract large context features. Extensive experiments demonstrate that GM-FlowNet outperforms RAFT, the most popular optimization-only method, by a large margin and achieves state-of-the-art performance on standard benchmarks. Thanks to the matching and overlapping attention, GMFlowNet obtains major improvements on the predictions for textureless regions and large motions. Our code is made publicly available at https://github.com/xiaofeng94/GMFlowNet.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation2015
- EpicFlow: Edge-preserving interpolation of correspondences for optical flow2015
- Large Displacement Optical Flow: Descriptor Matching in Variational Motion EstimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- A Duality Based Approach for Realtime TV-L 1 Optical Flow2007
- Highly Accurate Optic Flow Computation with Theoretically Justified WarpingInternational Journal of Computer Vision, 2006
- Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow MethodsInternational Journal of Computer Vision, 2005
- Combining the Advantages of Local and Global Optic Flow Methods2002
- Determining optical flowArtificial Intelligence, 1981