DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8915-8924
Abstract
Recently, the dense correlation volume method achieves state-of-the-art performance in optical flow. However, the correlation volume computation requires a lot of memory, which makes prediction difficult on high-resolution images. In this paper, we propose a novel Patchmatch-based framework to work on high-resolution optical flow estimation. Specifically, we introduce the first end-to-end Patchmatch based deep learning optical flow. It can get high-precision results with lower memory benefiting from propagation and local search of Patchmatch. Furthermore, a new inverse propagation is proposed to decouple the complex operations of propagation, which can significantly reduce calculations in multiple iterations. At the time of submission, our method ranks 1st on all the metrics on the popular KITTI2015 [28] benchmark, and ranks 2 nd on EPE on the Sintel [7] clean benchmark among published optical flow methods. Experiment shows our method has a strong cross-dataset generalization ability that the F1-all achieves 13.73%, reducing 21% from the best published result 17.4% on KITTI2015. What's more, our method shows a good details preserving result on the high-resolution dataset DAVIS [1] and consumes 2× less memory than RAFT [36]. Code will be available at github.com/zihuarheng/DIPKeywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Video Segmentation via Object Flow2016
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- DeepMatching: Hierarchical Deformable Dense MatchingInternational Journal of Computer Vision, 2016
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- Optical flow for self-supervised learning of obstacle appearance2015
- JOINT 3D ESTIMATION OF VEHICLES AND SCENE FLOWISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015
- Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow2014
- Topology-Constrained Layered Tracking with Latent Flow2013
- PatchMatchACM Transactions on Graphics, 2009
- Determining optical flowArtificial Intelligence, 1981