Learning to Reconstruct Confocal Microscopy Stacks from Single Light Field Images
Preprint
Abstract
We present a novel deep learning approach to reconstruct confocal microscopy stacks from single light field images. To perform the reconstruction, we introduce the LFMNet, a novel neural network architecture inspired by the U-Net design. It is able to reconstruct with high-accuracy a 112x112x57.6$\mu m^3$ volume (1287x1287x64 voxels) in 50ms given a single light field image of 1287x1287 pixels, thus dramatically reducing 720-fold the time for confocal scanning of assays at the same volumetric resolution and 64-fold the required storage. To prove the applicability in life sciences, our approach is evaluated both quantitatively and qualitatively on mouse brain slices with fluorescently labelled blood vessels. Because of the drastic reduction in scan time and storage space, our setup and method are directly applicable to real-time in vivo 3D microscopy. We provide analysis of the optical design, of the network architecture and of our training procedure to optimally reconstruct volumes for a given target depth range. To train our network, we built a data set of 362 light field images of mouse brain blood vessels and the corresponding aligned set of 3D confocal scans, which we use as ground truth. The data set will be made available for research purposes.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Snapshot Projection Optical TomographyPhysical Review Applied, 2020
- Diffraction-Assisted Light Field Microscopy for Microtomography and Digital Volume Correlation with Improved Spatial ResolutionExperimental Mechanics, 2019
- Instantaneous isotropic volumetric imaging of fast biological processesNature Methods, 2019
- High-speed volumetric imaging of neuronal activity in freely moving rodentsNature Methods, 2018
- Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopyNature Methods, 2017
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopyNature Methods, 2014
- The Light Field Camera: Extended Depth of Field, Aliasing, and SuperresolutionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2011
- Computer Control of Microscopes Using µManagerCurrent Protocols in Molecular Biology, 2010