Toward Interactive Self-Supervised Denoising
- 6 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (10) , 5360-5374
Abstract
Self-supervised denoising frameworks have recently been proposed to learn denoising models without noisy-clean image pairs, showing great potential in various applications. The denoising model is expected to produce visually pleasant images without noise patterns. However, it is non-trivial to achieve this goal using self-supervised methods because 1) the self-supervised model is difficult to restore the perceptual information due to the lack of clean supervision, and 2) perceptual quality is relatively subjective to users’ preferences. In this paper, we make the first attempt to build an interactive self-supervised denoising model to tackle the aforementioned problems. Specifically, we propose an interactive two-branch network to effectively restore perceptual information. The network consists of a denoising branch and an interactive branch, where the former focuses on efficient denoising, and the latter modulates the denoising branch. Based on the delicate architecture design, our network can produce various denoising outputs, allowing the user to easily select the most appealing outcome for satisfying the perceptual requirement. Moreover, to optimize the network with only noisy images, we propose a novel two-stage training strategy in a self-supervised way. Once the network is optimized, it can be interactively changed between noise reduction and texture restoration, providing more denoising choices for users. Existing self-supervised denoising methods can be integrated into our method to be user-friendly with interaction. Extensive experiments and comprehensive analyses are conducted to validate the effectiveness of the proposed method.Keywords
Funding Information
- National Natural Science Foundation of China (62131003, 62021001)
This publication has 47 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Why Deep Learning Works: A Manifold Disentanglement PerspectiveIEEE Transactions on Neural Networks and Learning Systems, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Image Restoration Using Joint Statistical Modeling in a Space-Transform DomainIEEE Transactions on Circuits and Systems for Video Technology, 2014
- Non-Local Means DenoisingImage Processing On Line, 2011
- Image Denoising by Sparse 3-D Transform-Domain Collaborative FilteringIEEE Transactions on Image Processing, 2007
- Unsupervised Learning of Image Manifolds by Semidefinite ProgrammingInternational Journal of Computer Vision, 2006
- An information fidelity criterion for image quality assessment using natural scene statisticsIEEE Transactions on Image Processing, 2005
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics2002
- The influence of the sigmoid function parameters on the speed of backpropagation learning1995