DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework
Preprint
Abstract
Optical flow estimation is essential for video processing tasks, such as restoration and action recognition. The quality of videos is constantly increasing, with current standards reaching 8K resolution. However, optical flow methods are usually designed for low resolution and do not generalize to large inputs due to their rigid architectures. They adopt downscaling or input tiling to reduce the input size, causing a loss of details and global information. There is also a lack of optical flow benchmarks to judge the actual performance of existing methods on high-resolution samples. Previous works only conducted qualitative high-resolution evaluations on hand-picked samples. This paper fills this gap in optical flow estimation in two ways. We propose DPFlow, an adaptive optical flow architecture capable of generalizing up to 8K resolution inputs while trained with only low-resolution samples. We also introduce Kubric-NK, a new benchmark for evaluating optical flow methods with input resolutions ranging from 1K to 8K. Our high-resolution evaluation pushes the boundaries of existing methods and reveals new insights about their generalization capabilities. Extensive experimental results show that DPFlow achieves state-of-the-art results on the MPI-Sintel, KITTI 2015, Spring, and other high-resolution benchmarks.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow2024
- Lightweight Optical Flow Estimation Using 1D MatchingIEEE Access, 2024
- MS-RAFT+: High Resolution Multi-Scale RAFTInternational Journal of Computer Vision, 2023
- FlowFormer: A Transformer Architecture for Optical Flow2022
- RAFT: Recurrent All-Pairs Field Transforms for Optical Flow2020
- On the Integration of Optical Flow and Action Recognition2019
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Object scene flow for autonomous vehicles2015
- A Naturalistic Open Source Movie for Optical Flow Evaluation2012
- High Accuracy Optical Flow Estimation Based on a Theory for Warping2004