Towards Robust and Generalizable Lensless Imaging With Modular Learned Reconstruction
- 28 February 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Imaging
- Vol. 11 (25730436) , 213-227
Abstract
Lensless cameras disregard the conventional design that imaging should mimic the human eye. This is done by replacing the lens with a thin mask, and moving image formation to the digital post-processing. State-of-the-art lensless imaging techniques use learned approaches that combine physical modeling and neural networks. However, these approaches make simplifying modeling assumptions for ease of calibration and computation. Moreover, the generalizability of learned approaches to lensless measurements of new masks has not been studied. To this end, we utilize a modular learned reconstruction in which a key component is a pre-processor prior to image recovery. We theoretically demonstrate the pre-processor's necessity for standard image recovery techniques (Wiener filtering and iterative algorithms), and through extensive experiments show its effectiveness for multiple lensless imaging approaches and across datasets of different mask types (amplitude and phase). We also perform the first generalization benchmark across mask types to evaluate how well reconstructions trained with one system generalize to others. Our modular reconstruction enables us to use pre-trained components and transfer learning on new systems to cut down weeks of tedious measurements and training. As part of our work, we open-source four datasets, and software for measuring datasets and for training our modular reconstruction.Keywords
All Related Versions
Funding Information
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (CRSII5_213521)
- DigiLight—Programmable Third-Harmonic Generation Microscopy Applied to Advanced Manufacturing
This publication has 43 references indexed in Scilit:
- Restormer: Efficient Transformer for High-Resolution Image Restoration2022
- In vivo lensless microscopy via a phase mask generating diffraction patterns with high-contrast contoursNature Biomedical Engineering, 2022
- Deep Learning for Handling Kernel/model Uncertainty in Image Deconvolution2020
- Single-shot lensless imaging with fresnel zone aperture and incoherent illuminationLight: Science & Applications, 2020
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Lensless Imaging with a Controllable Aperture2006
- Coded aperture imaging in X- and gamma-ray astronomySpace Science Reviews, 1987
- Introduction to Fourier OpticsPhysics Today, 1969
- Scatter-Hole Cameras for X-Rays and Gamma RaysThe Astrophysical Journal, 1968