Learning Accurate Dense Correspondences and When to Trust Them
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5710-5720
Abstract
Establishing dense correspondences between a pair of images is an important and general problem. However, dense flow estimation is often inaccurate in the case of large displacements or homogeneous regions. For most applications and down-stream tasks, such as pose estimation, image manipulation, or 3D reconstruction, it is crucial to know when and where to trust the estimated matches.In this work, we aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map indicating the reliability and accuracy of the prediction. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. In particular, we parametrize the predictive distribution as a constrained mixture model, ensuring better modelling of both accurate flow predictions and outliers. Moreover, we develop an architecture and training strategy tailored for robust and generalizable uncertainty prediction in the context of self-supervised training. Our approach obtains state- of-the-art results on multiple challenging geometric matching and optical flow datasets. We further validate the usefulness of our probabilistic confidence estimation for the task of pose estimation. Code and models are available at https://github.com/PruneTruong/PDCNet.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind ThemInternational Journal of Computer Vision, 2013
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Bootstrap optical flow confidence and uncertainty measureComputer Vision and Image Understanding, 2011
- Non-rigid dense correspondence with applications for image enhancementACM Transactions on Graphics, 2011
- A Database and Evaluation Methodology for Optical FlowInternational Journal of Computer Vision, 2010
- A Statistical Confidence Measure for Optical Flows2008
- An Adaptive Confidence Measure for Optical Flows Based on Linear Subspace Projections2007
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Performance of optical flow techniquesInternational Journal of Computer Vision, 1994
- “Determining optical flow”: a retrospectiveArtificial Intelligence, 1993