MaskRNN: Instance Level Video Object Segmentation
Preprint
Abstract
Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance -- a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- FusionSeg: Learning to Combine Motion and Appearance for Fully Automatic Segmentation of Generic Objects in Videos2017
- Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions2013
- LIVEcut: Learning-based interactive video segmentation by evaluation of multiple propagated cues2009
- Detecting Motion Regions in the Presence of a Strong Parallax from a Moving Camera by Multiview Geometric ConstraintsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Statistical background modeling for non-stationary cameraPattern Recognition Letters, 2003
- Background and foreground modeling using nonparametric kernel density estimation for visual surveillanceProceedings of the IEEE, 2002
- Shape matching and object recognition using shape contextsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Concerning Bayesian motion segmentation, model averaging, matching and the trifocal tensor1998
- A unified approach to moving object detection in 2D and 3D scenes1996
- Computing occluding and transparent motionsInternational Journal of Computer Vision, 1994