Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network
Authors:
- Jeffrey AlidoJAJeffrey Alido
- Joseph GreeneJGJoseph Greene
- Yujia XueYXYujia Xue
- Guorong HuGHGuorong Hu
- Mitchell GilmoreMGMitchell Gilmore
- Lei TianLTLei Tian
- Yunzhe LiYLYunzhe Li
- 6 February 2024
- journal article
- research article
- Published by Optica Publishing Group in Optics Express
- Vol. 32 (4) , 6241-6257
Abstract
Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed computational miniature mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model’s generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.Keywords
Funding Information
- National Institutes of Health (R01NS126596)
- National Science Foundation (2234657)
This publication has 30 references indexed in Scilit:
- Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic dataNature Methods, 2023
- Adaptive 3D descattering with a dynamic synthesis networkLight: Science & Applications, 2022
- Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopyNature Communications, 2021
- Miniscope3D: optimized single-shot miniature 3D fluorescence microscopyLight: Science & Applications, 2020
- Cortical Observation by Synchronous Multifocal Optical Sampling Reveals Widespread Population Encoding of ActionsNeuron, 2020
- Readout of fluorescence functional signals through highly scattering tissueNature Photonics, 2020
- Learning to synthesize: robust phase retrieval at low photon countsLight: Science & Applications, 2020
- High-speed volumetric imaging of neuronal activity in freely moving rodentsNature Methods, 2018
- Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopyNature Methods, 2017
- In vivo three-photon microscopy of subcortical structures within an intact mouse brainNature Photonics, 2013