Video object segmentation via couple streams and feature memory
Open Access
- 17 April 2024
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 18 (9) , 2257-2272
Abstract
In recent years, most video segmentation methods use deep CNN to process the input image, but they did not fully mine the rich intermediate predictions in spatio‐temporal space. And, the segmentation challenges such as occlusion, severe deformation and illumination have not been well solved so far. To alleviate these problems, this paper focuses on constructing multi module network structures that represent multi semantics and proposes a video object segmentation network via coupled‐stream architecture with feature memory mechanism. This network first extracts high‐level semantic features, edge features, long‐term and short‐term stable depth features of the target, and then decode them into the segmentation mask of target. In addition, negative skeleton inhibition and frame interpolation are used to prevent the interference of similar objects and motion blur, respectively. The method has a low GPU memory usage, regardless of the number of object in video. And performs 86.5%and 62.4% in J&F measure on DAVIS 2016 and DAVIS 2017 validation set, without fine‐tuning and online training.Keywords
Funding Information
- National Natural Science Foundation of China (61772209)
- Science and Technology Planning Project of Guangdong Province (2019A050510034)
This publication has 51 references indexed in Scilit:
- AGUnet: Annotation-guided U-net for fast one-shot video object segmentationPattern Recognition, 2021
- Kernelized Memory Network for Video Object Segmentation2020
- Collaborative Video Object Segmentation by Foreground-Background Integration2020
- Higher-order potentials for video object segmentation in bilateral spaceNeurocomputing, 2020
- PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation2019
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018
- Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- SeamSeg: Video Object Segmentation Using Patch Seams2014
- ImageNet: A large-scale hierarchical image database2009