Abstract
While identifying object instances and accurately reconstructing their geometry are fundamental to manipulation, passive visual observation in static scenes is inherently constrained by occlusions, which gives rise to both geometric and semantic ambiguities, particularly in complex multi-object settings. We present INGRID (INteractive GeometRy and instance IDentification), a system in which a robotic manipulator actively interacts with the scene to reveal previously hidden surfaces and disambiguate object instances and geometries. The key challenge lies first in deciding where to interact in order to induce the most informative scene changes. A subsequent challenge is then to efficiently update both the object instances and their geometries once new visual observations become available. To address these challenges, INGRID follows a four-step pipeline: (i) constructing an instance candidate tree, (ii) optimizing informative actions, (iii) identifying instances under rigid-body assumptions, and (iv) selectively finetuning geometry. Together, these steps enable efficient, interaction-driven geometry and instance identification. Our method proves robust and practical in complex multi-object scenes with severe occlusions, both in simulation and in the real world.

This publication has 16 references indexed in Scilit: