IVA-former: invisible–visible query guided amodal mask measurement network for desktop object via hierarchical transformer
- 22 October 2024
- journal article
- research article
- Published by IOP Publishing in Measurement Science and Technology
- Vol. 36 (1) , 015410
Abstract
Instance segmentation of desktop objects is important for service robots. Most of the previous works for desktop environments are restricted to measuring the visible area of target objects. However, when a target object is placed behind another, the algorithm that only performs visible area segmentation is unable to provide accurate appearance information for the occluded objects. To solve this problem, we propose the invisible–visible query guided amodal mask measurement network based on a hierarchical transformer for desktop scenes, which can perceive the entire appearance of objects in the presence of occlusions. In this method, the RGB-D backbone is adopted to fuse the features from both RGB and depth images. Then, the pixel decoder is used to generate multi-scale feature maps. We then adopt a hierarchical transformer decoder to predict invisible, visible, and amodal masks simultaneously. To enhance the associations between the three prediction branches, we propose a query transform module to transfer object queries between adjacent branches. Since amodal masks are a combination of invisible and visible masks, we propose an invisible–visible mixture loss that takes masks from both invisible and visible branches to further supervise the network. Our method is trained on synthetic datasets for desktop objects and evaluated on both visible and amodal real-world datasets. Compared to other recent segmentation algorithms, our method achieves competitive performance.Keywords
Funding Information
- Major Science and Technology Projects of Liaoning Province (No. 2021JH1/10400049)
- National Natural Science Foundation of China (No. 61973066)
This publication has 47 references indexed in Scilit:
- 6D pose estimation of object based on fused region-level feature in cluttered scenesMeasurement Science and Technology, 2023
- Self-distilled Vision Transformer for Domain Generalization2023
- 2D Amodal Instance Segmentation Guided by 3D Shape Prior2022
- Inferring Occluded Geometry Improves Performance When Retrieving an Object from Dense Clutter2022
- Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation2020
- End-to-End Object Detection with Transformers2020
- Inward-region-growing-based accurate partitioning of closely stacked objects for bin-pickingMeasurement Science and Technology, 2020
- BOP Challenge 2020 on 6D Object Localization2020
- Amodal Instance Segmentation2016
- Microsoft COCO: Common Objects in Context2014