Abstract
Traditional perspective image inpainting tasks focus on restoring regions using a random mask image. However, the emergence of virtual reality (VR) renders traditional image inpainting inadequate for interactive editing, as users demand the ability to precisely remove specific objects in VR scenes rather than relying on arbitrary masks. Moreover, existing perspective inpainting methods exhibit inherent weaknesses in maintaining geometric distortions and photometric consistency in spherical projections. In this paper, we introduce a new task for interactively removing a queried object and subsequently repairing the missing regions in indoor $360^{\circ }$ images. Moreover, we propose a robust baseline methodology for indoor $360^{\circ }$ object removal that integrates coordinate querying with spherical normal constraints and learnable wavelet transform to enhance the three-dimensionality and lighting fidelity of repaired $360^{\circ }$ images. Specifically, we develop a spherical normal estimation module that takes advantage of the spherical projection properties of $360^{\circ }$ images to generate accurate surface normals for constraining the spherical geometry of the restored $360^{\circ }$ images. Subsequently, we design a wavelet-enhanced spatial feature aggregation module to improve directional continuity and frequency characteristics in restored regions, enabling the repaired $360 ^{\circ }$ images to exhibit clearer textures and sharper edges. Furthermore, to address the high-resolution nature of $360^{\circ }$ images, a mask region pixel coordinate querying module is introduced to alleviate memory constraints inherent in explicit $360 ^{\circ }$ image representations, enabling high-resolution content synthesis for any designated region. We also establish a new object removal dataset for indoor $360^{\circ }$ images (S3D-Removal), comprising 18k+ pairs of $360^{\circ }$ images with corresponding query points. Extensive experimental results on the S3D-Removal dataset demonstrate that our proposed method consistently outperforms state-of-the-art inpainting techniques across all evaluation metrics.
Funding Information
  • National Natural Science Foundation of China (62371310)
  • Shenzhen Science and Technology Program (JCYJ20241202124415021)
  • Guangdong Basic and Applied Basic Research Foundation (2026A1515010230)
  • Stable Support Project of Shenzhen (20231122122722001)
  • Scientific Foundation for Youth Scholars of Shenzhen University