Spherical Image Inpainting with Frame Transformation and Data-Driven Prior Deep Networks
- 24 July 2023
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Imaging Sciences
- Vol. 16 (3) , 1177-1194
Abstract
Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modeling, and medical imaging. It is nontrivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with a deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regularizer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using a deep learning denoiser and a plug-and-play model.Keywords
All Related Versions
Funding Information
- National Key Research & Development Program of China (2021YFE0203700)
- Hong Kong Research Grants Council (14300219, 14302920, 14301121, 12300218, 12300519, 17201020, 17300021, C1013-21GF, C7004- 21GF, CUHK14301718, C1013-21GF, CityU11301120, 9380101)
- Chinese University of Hong Kong (N CUHK 415/19, N-HKU76921)
- Innovation and Technology Fund (MHP/038/20)
- National Natural Science Foundation of China (11871210, 11971215, 61971292)
This publication has 28 references indexed in Scilit:
- Deep plug-and-play prior for low-rank tensor completionNeurocomputing, 2020
- Soft-Edge Assisted Network for Single Image Super-ResolutionIEEE Transactions on Image Processing, 2020
- Reconstruction of the High Resolution Phase in a Closed Loop Adaptive Optics SystemSIAM Journal on Imaging Sciences, 2020
- Multilinear Compressive Sensing and an Application to Convolutional Linear NetworksSIAM Journal on Mathematics of Data Science, 2019
- Fixing Nonconvergence of Algebraic Iterative Reconstruction with an Unmatched BackprojectorSIAM Journal on Scientific Computing, 2019
- Multi-shell diffusion signal recovery from sparse measurementsMedical Image Analysis, 2014
- Toward mapping the effective elastic thickness of planetary lithospheres from a spherical wavelet analysis of gravity and topographyPhysics of the Earth and Planetary Interiors, 2014
- Multichannel Poisson denoising and deconvolution on the sphere: application to theFermiGamma-ray Space TelescopeAstronomy & Astrophysics, 2012
- Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneityGeophysical Journal International, 2011
- Cosmological Applications of a Wavelet Analysis on the SphereJournal of Fourier Analysis and Applications, 2007