Salient Object Detection in Stereoscopic 3D Images Using a Deep Convolutional Residual Autoencoder
- 21 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 3388-3399
Abstract
In recent years, the detection of distinctive objects in stereoscopic 3D images has drawn increasing attention. Unlike 2D salient object detection, salient object detection in stereoscopic 3D images is highly challenging. Hence, we propose a novel Deep Convolutional Residual Autoencoder (DCRA) for end-to-end salient object detection in stereoscopic 3D images. The core trainable architecture of the salient object detection model employs raw stereoscopic 3D images as the inputs and their corresponding ground truth saliency masks as the labels. A convolutional residual module is applied to both the encoder and the decoder as a basic building block in the DCRA, and long-range skip connections are employed to bypass the equal-sized feature maps between the encoder and the decoder. To explore the complex relationships and exploit the complementarity between RGB (photometric) and depth (geometric) information, multiple feature map fusion modules are constructed. These modules integrate texture and structure information between the RGB and depth branches of the encoder and fuse their features over several multiscale layers. Finally, to efficiently optimize DCRA parameters, a supervision pyramid based on boundary loss and background prior loss is adopted, which employs supervised learning over the multiscale layers in the decoder to prevent vanishing gradients and accelerate the training at the fusion stage. We compare the proposed DCRA with state-of-the-art methods on two challenging benchmark datasets. The results of these experiments demonstrate that our proposed DCRA performs favorably against the comparison models.Keywords
Funding Information
- National Natural Science Foundation of China (61502429, 61672337, 61972357)
- Zhejiang Provincial Natural Science Foundation of China (LY18F020012)
This publication has 60 references indexed in Scilit:
- Depth-aware salient object detection using anisotropic center-surround differenceSignal Processing: Image Communication, 2015
- Salient object detection in RGB-D image based on saliency fusion and propagation2015
- Inner and Inter Label Propagation: Salient Object Detection in the WildIEEE Transactions on Image Processing, 2015
- Background Prior-Based Salient Object Detection via Deep Reconstruction ResidualIEEE Transactions on Circuits and Systems for Video Technology, 2014
- Global Contrast Based Salient Region DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Depth Enhanced Saliency Detection Method2014
- Saliency Detection for Stereoscopic ImagesIEEE Transactions on Image Processing, 2014
- Enhanced Computer Vision With Microsoft Kinect Sensor: A ReviewIEEE Transactions on Cybernetics, 2013
- Depth really Matters: Improving Visual Salient Region Detection with Depth2013
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998