INGRID:Interactive Geometry and Instance Identification for Occluded Scenes
Authors:
- Junho LeeJLJunho Lee
- Sang Min KimSKSang Min Kim
- Yonghyeon LeeYLYonghyeon Lee
- Young Min KimYKYoung Min Kim
Preprint
- 15 September 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
While identifying object instances and accurately reconstructing their geometry are fundamental to manipulation, passive visual observation in static scenes is inherently constrained by occlusions, which gives rise to both geometric and semantic ambiguities, particularly in complex multi-object settings. We present INGRID (INteractive GeometRy and instance IDentification), a system in which a robotic manipulator actively interacts with the scene to reveal previously hidden surfaces and disambiguate object instances and geometries. The key challenge lies first in deciding where to interact in order to induce the most informative scene changes. A subsequent challenge is then to efficiently update both the object instances and their geometries once new visual observations become available. To address these challenges, INGRID follows a four-step pipeline: (i) constructing an instance candidate tree, (ii) optimizing informative actions, (iii) identifying instances under rigid-body assumptions, and (iv) selectively finetuning geometry. Together, these steps enable efficient, interaction-driven geometry and instance identification. Our method proves robust and practical in complex multi-object scenes with severe occlusions, both in simulation and in the real world.Keywords
This publication has 16 references indexed in Scilit:
- NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM2024
- NFL: Normal Field Learning for 6-DoF Grasping of Transparent ObjectsIEEE Robotics and Automation Letters, 2023
- AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal DomainsIEEE Transactions on Robotics, 2023
- GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF2023
- Vision-Only Robot Navigation in a Neural Radiance WorldIEEE Robotics and Automation Letters, 2022
- Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a reviewArtificial Intelligence Review, 2020
- Grasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost DemonstrationsIEEE Robotics and Automation Letters, 2020
- Interactive Perception: Leveraging Action in Perception and Perception in ActionIEEE Transactions on Robotics, 2017
- Active visionInternational Journal of Computer Vision, 1988
- Active perceptionProceedings of the IEEE, 1988