Occlusion relationship reasoning with a feature separation and interaction network
Open Access
- 13 October 2023
- journal article
- research article
- Published by Springer Nature in Visual Intelligence
- Vol. 1 (1) , 1-16
Abstract
Occlusion relationship reasoning aims to locate where an object occludes others and estimate the depth order of these objects in three-dimensional (3D) space from a two-dimensional (2D) image. The former sub-task demands both the accurate location and the semantic indication of the objects, while the latter sub-task needs the depth order among the objects. Although several insightful studies have been proposed, a key characteristic of occlusion relationship reasoning, i.e., the specialty and complementarity between occlusion boundary detection and occlusion orientation estimation, is rarely discussed. To verify this claim, in this paper, we integrate these properties into a unified end-to-end trainable network, namely the feature separation and interaction network (FSINet). It contains a shared encoder-decoder structure to learn the complementary property between the two sub-tasks, and two separated paths to learn specialized properties of the two sub-tasks. Concretely, the occlusion boundary path contains an image-level cue extractor to capture rich location information of the boundary, a detail-perceived semantic feature extractor, and a contextual correlation extractor to acquire refined semantic features of objects. In addition, a dual-flow cross detector has been customized to alleviate false-positive and false-negative boundaries. For the occlusion orientation estimation path, a scene context learner has been designed to capture the depth order cue around the boundary. In addition, two stripe convolutions are built to judge the depth order between objects. The shared decoder supplies the feature interaction, which plays a key role in exploiting the complementarity of the two paths. Extensive experimental results on the PIOD and BSDS ownership datasets reveal the superior performance of FSINet over state-of-the-art alternatives. Additionally, abundant ablation studies are offered to demonstrate the effectiveness of our design.Keywords
Funding Information
- National Natural Science Foundation of China (62176098)
- National Natural Science Foundation of China (61703049)
- Natural Science Foundation of Hubei Province (2019CFA022)
This publication has 43 references indexed in Scilit:
- Monocular Depth-Ordering Reasoning with Occlusion Edge Detection and Couple Layers InferenceIEEE Intelligent Systems, 2016
- Monocular Object Instance Segmentation and Depth Ordering with CNNs2015
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Feedforward semantic segmentation with zoom-out features2015
- Fast 2D border ownership assignment2015
- Monocular Extraction of 2.1D Sketch Using Constrained Convex OptimizationInternational Journal of Computer Vision, 2014
- Recovering Relative Depth from Low-Level Features Without Explicit T-junction Detection and InterpretationInternational Journal of Computer Vision, 2013
- A learning-based framework for depth ordering2012
- A segmentation-aware object detection model with occlusion handling2011
- Recovering Occlusion Boundaries from an ImageInternational Journal of Computer Vision, 2010