Learning Video Object Segmentation from Static Images
Authors:
- Federico PerazziFPFederico Perazzi
- Anna KhorevaAKAnna Khoreva
- Rodrigo BenensonRBRodrigo Benenson
- Bernt SchieleBSBernt Schiele
- Alexander Sorkine-HornungASAlexander Sorkine-Hornung
Top Cited Papers
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 3491-3500
Abstract
Inspired by recent advances of deep learning in instance segmentation and object tracking, we introduce the concept of convnet-based guidance applied to video object segmentation. Our model proceeds on a per-frame basis, guided by the output of the previous frame towards the object of interest in the next frame. We demonstrate that highly accurate object segmentation in videos can be enabled by using a convolutional neural network (convnet) trained with static images only. The key component of our approach is a combination of offline and online learning strategies, where the former produces a refined mask from the previous frame estimate and the latter allows to capture the appearance of the specific object instance. Our method can handle different types of input annotations such as bounding boxes and segments while leveraging an arbitrary amount of annotated frames. Therefore our system is suitable for diverse applications with different requirements in terms of accuracy and efficiency. In our extensive evaluation, we obtain competitive results on three different datasets, independently from the type of input annotation.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- JumpCutACM Transactions on Graphics, 2015
- Hierarchical Image Saliency Detection on Extended CSSDIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Global Contrast Based Salient Region DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- TouchCut: Fast image and video segmentation using single-touch interactionComputer Vision and Image Understanding, 2014
- Discontinuity-aware video object cutoutACM Transactions on Graphics, 2012
- Real-time edge-aware image processing with the bilateral gridACM Transactions on Graphics, 2007
- "GrabCut"ACM Transactions on Graphics, 2004
- Principal warps: thin-plate splines and the decomposition of deformationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1989