Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
Authors:
- Yuhua ChenYCYuhua Chen
- Jordi Pont-TusetJPJordi Pont-Tuset
- Alberto MontesAMAlberto Montes
- Luc Van GoolLGLuc Van Gool
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1189-1198
Abstract
This paper tackles the problem of video object segmentation, given some user annotation which indicates the object of interest. The problem is formulated as pixel-wise retrieval in a learned embedding space: we embed pixels of the same object instance into the vicinity of each other, using a fully convolutional network trained by a modified triplet loss as the embedding model. Then the annotated pixels are set as reference and the rest of the pixels are classified using a nearest-neighbor approach. The proposed method supports different kinds of user input such as segmentation mask in the first frame (semi-supervised scenario), or a sparse set of clicked points (interactive scenario). In the semi-supervised scenario, we achieve results competitive with the state of the art but at a fraction of computation cost (275 milliseconds per frame). In the interactive scenario where the user is able to refine their input iteratively, the proposed method provides instant response to each input, and reaches comparable quality to competing methods with much less interaction.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Fully Connected Object Proposals for Video Segmentation2015
- Unsupervised Learning of Visual Representations Using Videos2015
- Video Segmentation with Just a Few Strokes2015
- JumpCutACM Transactions on Graphics, 2015
- FaceNet: A unified embedding for face recognition and clustering2015
- SeamSeg: Video Object Segmentation Using Patch Seams2014
- TouchCut: Fast image and video segmentation using single-touch interactionComputer Vision and Image Understanding, 2014
- Learning a Similarity Metric Discriminatively, with Application to Face Verification2005
- Interactive video cutoutACM Transactions on Graphics, 2005
- Content-based image retrieval at the end of the early yearsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000