Deep LoRA-Unfolding Networks for Image Restoration
Authors:
- Xiangming WangXWXiangming Wang
- Haijin ZengHZHaijin Zeng
- Benteng SunBSBenteng Sun
- Jiezhang CaoJCJiezhang Cao
- Kai ZhangKZKai Zhang
- Qiangqiang ShenQSQiangqiang Shen
- Yongyong ChenYCYongyong Chen
Preprint
Abstract
Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and super-resolution.It unfolds the iterative optimization steps into a stack of sequentially linked blocks.Each block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known level.However, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained scenarios.To address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient DUN.LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step.This design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$-stage DUN with on-par or better performance.Extensive experiments conducted on three IR tasks validate the efficiency of our method.
Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- Deep Unfolding for Snapshot Compressive ImagingInternational Journal of Computer Vision, 2023
- Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction2022
- Multi-Stage Progressive Image Restoration2021
- Hyperspectral image denoising via global spatial-spectral total variation regularized nonconvex local low-rank tensor approximationSignal Processing, 2021
- End-to-End Low Cost Compressive Spectral Imaging with Spatial-Spectral Self-Attention2020
- Cascaded Deep Video Deblurring Using Temporal Sharpness Prior2020
- ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing2018
- ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Microsoft COCO: Common Objects in Context2014