Interactive $360 ^{\circ }$ Object Removal: A Coordinate Querying Approach via Spherical Normal Constraints and Learnable Wavelet Enhancement
- 29 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-13
Abstract
Traditional perspective image inpainting tasks focus on restoring regions using a random mask image. However, the emergence of virtual reality (VR) renders traditional image inpainting inadequate for interactive editing, as users demand the ability to precisely remove specific objects in VR scenes rather than relying on arbitrary masks. Moreover, existing perspective inpainting methods exhibit inherent weaknesses in maintaining geometric distortions and photometric consistency in spherical projections. In this paper, we introduce a new task for interactively removing a queried object and subsequently repairing the missing regions in indoor $360^{\circ }$ images. Moreover, we propose a robust baseline methodology for indoor $360^{\circ }$ object removal that integrates coordinate querying with spherical normal constraints and learnable wavelet transform to enhance the three-dimensionality and lighting fidelity of repaired $360^{\circ }$ images. Specifically, we develop a spherical normal estimation module that takes advantage of the spherical projection properties of $360^{\circ }$ images to generate accurate surface normals for constraining the spherical geometry of the restored $360^{\circ }$ images. Subsequently, we design a wavelet-enhanced spatial feature aggregation module to improve directional continuity and frequency characteristics in restored regions, enabling the repaired $360 ^{\circ }$ images to exhibit clearer textures and sharper edges. Furthermore, to address the high-resolution nature of $360^{\circ }$ images, a mask region pixel coordinate querying module is introduced to alleviate memory constraints inherent in explicit $360 ^{\circ }$ image representations, enabling high-resolution content synthesis for any designated region. We also establish a new object removal dataset for indoor $360^{\circ }$ images (S3D-Removal), comprising 18k+ pairs of $360^{\circ }$ images with corresponding query points. Extensive experimental results on the S3D-Removal dataset demonstrate that our proposed method consistently outperforms state-of-the-art inpainting techniques across all evaluation metrics.Keywords
Funding Information
- National Natural Science Foundation of China (62371310)
- Shenzhen Science and Technology Program (JCYJ20241202124415021)
- Guangdong Basic and Applied Basic Research Foundation (2026A1515010230)
- Stable Support Project of Shenzhen (20231122122722001)
- Scientific Foundation for Youth Scholars of Shenzhen University
This publication has 33 references indexed in Scilit:
- An end-to-end shadow removal framework with an intuitive interaction schemePattern Recognition, 2025
- PanoFormer: Panorama Transformer for Indoor 360$^{\circ }$ Depth Estimation2022
- Structured3D: A Large Photo-Realistic Dataset for Structured 3D Modeling2020
- Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations2020
- Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting2020
- Deep Normal Estimation for Automatic Shading of Hand-Drawn Characters2019
- High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs2018
- GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation2018
- Matterport3D: Learning from RGB-D Data in Indoor Environments2017
- On the Canny edge detectorPattern Recognition, 2001