Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories
Preprint
Abstract
Tracking pixels in videos is typically studied as an optical flow estimation problem, where every pixel is described with a displacement vector that locates it in the next frame. Even though wider temporal context is freely available, prior efforts to take this into account have yielded only small gains over 2-frame methods. In this paper, we revisit Sand and Teller's "particle video" approach, and study pixel tracking as a long-range motion estimation problem, where every pixel is described with a trajectory that locates it in multiple future frames. We re-build this classic approach using components that drive the current state-of-the-art in flow and object tracking, such as dense cost maps, iterative optimization, and learned appearance updates. We train our models using long-range amodal point trajectories mined from existing optical flow data that we synthetically augment with multi-frame occlusions. We test our approach in trajectory estimation benchmarks and in keypoint label propagation tasks, and compare favorably against state-of-the-art optical flow and feature tracking methods.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- MAST: A Memory-Augmented Self-Supervised Tracker2020
- Learning Correspondence From the Cycle-Consistency of Time2019
- Long-Term Tracking in the Wild: A Benchmark2018
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018
- PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume2018
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks2017
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow2010
- Particle Video: Long-Range Motion Estimation using Point Trajectories2006
- Stochastic Tracking of 3D Human Figures Using 2D Image Motion2000