Novel Dilated Separable Convolution Networks for Efficient Video Salient Object Detection in the Wild
- 7 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-13
Abstract
Appearance and motion are essential features in Video Salient Object Detection (VSOD) tasks. Most of the existing approaches utilize local features and thus fail to understand both the appearance as well as motion-specific semantics at the global level. Hence, these methods are unable to perform in unconstrained scenarios where multiple challenges, such as partial occlusion, motion blur, noise, clutter background, etc., exist. Moreover, these approaches require a large number of computational resources due to their complex structures, which limits their applicability to real-world deployment. To resolve these issues and to achieve a balance between accuracy and computational complexity, in this paper, a Dilation Separable Convolution Network (DSCNet) is proposed, which is equipped with Dilation Attention Fusion Module (DAFM), Bi-directional Cross-modality Fusion Module (BCFM), and Saliency Prediction Module (SPM) to extract enhanced multi-scaled motion and appearance features without increasing the model complexity. Further, a Bi-directional Separable Convolution Network (BSC-Net) equipped with a Separable Convolution Module (SCM)s and a FlowNet2.0 is proposed to utilize multi-scale contextual information across appearance cues and generate enhanced multi-scaled motion maps. For faster and better training of the DSCNet model, we propose a novel Stochastic Gradient-based Firefly Algorithm (SGFA), which adaptively balances the exploration and exploitation in multi-scaled, cross-modal embedded sub-spaces. With the help of the proposed SGFA algorithm, DSCNet+ model is constructed on top of DSCNet, which further improves the results in terms of the training speed as well as other evaluation metrics. The proposed models are evaluated on six benchmark datasets, and a detailed comparative study is provided with sixteen state-of-the-art (SOTA) models. One of the major highlights of the work is the significant performance of the proposed models on the most difficult DAVSOD-Diff dataset, which best reflects the challenging real-world scenarios.Keywords
This publication has 49 references indexed in Scilit:
- Unsupervised Video Object Segmentation Using Motion Saliency-Guided Spatio-Temporal Propagation2018
- Learning Video Object Segmentation with Visual Memory2017
- SegFlow: Joint Learning for Video Object Segmentation and Optical Flow2017
- Super-Trajectory for Video Segmentation2017
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks2017
- Learning Video Object Segmentation from Static Images2017
- 1 year, 1000 km: The Oxford RobotCar datasetThe International Journal of Robotics Research, 2016
- A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation2016
- Fully Connected Object Proposals for Video Segmentation2015
- Video Segmentation by Tracking Many Figure-Ground Segments2013