Modality-Guided Subnetwork for Salient Object Detection
Preprint
Abstract
Recent RGBD-based models for saliency detection have attracted research attention. The depth clues such as boundary clues, surface normal, shape attribute, etc., contribute to the identification of salient objects with complicated scenarios. However, most RGBD networks require multi-modalities from the input side and feed them separately through a two-stream design, which inevitably results in extra costs on depth sensors and computation. To tackle these inconveniences, we present in this paper a novel fusion design named modality-guided subnetwork (MGSnet). It has the following superior designs: 1) Our model works for both RGB and RGBD data, and dynamically estimating depth if not available. Taking the inner workings of depth-prediction networks into account, we propose to estimate the pseudo-geometry maps from RGB input - essentially mimicking the multi-modality input. 2) Our MGSnet for RGB SOD results in real-time inference but achieves state-of-the-art performance compared to other RGB models. 3) The flexible and lightweight design of MGS facilitates the integration into RGBD two-streaming models. The introduced fusion design enables a cross-modality interaction to enable further progress but with a minimal cost.Keywords
All Related Versions
This publication has 3 references indexed in Scilit:
- Spatial Information Guided Convolution for Real-Time RGBD Semantic SegmentationIEEE Transactions on Image Processing, 2021
- Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale BenchmarksIEEE Transactions on Neural Networks and Learning Systems, 2020
- SIFT Flow: Dense Correspondence across Scenes and Its ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010