Petascale pipeline for precise alignment of images from serial section electron microscopy
Open Access
- 4 January 2024
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 15 (1) , 1-15
Abstract
The reconstruction of neural circuits from serial section electron microscopy (ssEM) images is being accelerated by automatic image segmentation methods. Segmentation accuracy is often limited by the preceding step of aligning 2D section images to create a 3D image stack. Precise and robust alignment in the presence of image artifacts is challenging, especially as datasets are attaining the petascale. We present a computational pipeline for aligning ssEM images with several key elements. Self-supervised convolutional nets are trained via metric learning to encode and align image pairs, and they are used to initialize iterative fine-tuning of alignment. A procedure called vector voting increases robustness to image artifacts or missing image data. For speedup the series is divided into blocks that are distributed to computational workers for alignment. The blocks are aligned to each other by composing transformations with decay, which achieves a global alignment without resorting to a time-consuming global optimization. We apply our pipeline to a whole fly brain dataset, and show improved accuracy relative to prior state of the art. We also demonstrate that our pipeline scales to a cubic millimeter of mouse visual cortex. Our pipeline is publicly available through two open source Python packages.Keywords
All Related Versions
Funding Information
- ODNI | Intelligence Advanced Research Projects Activity (D16PC00004)
- ODNI | Intelligence Advanced Research Projects Activity (D16PC0005)
- U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (U19 NS104648)
- U.S. Department of Health & Human Services | NIH | National Eye Institute (R01 EY027036)
- U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (U01 MH114824)
- U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (R01NS104926)
- U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (RF1MH123400)
- ODNI | Intelligence Advanced Research Projects Activity (D16PC00003)
This publication has 34 references indexed in Scilit:
- ssEMnet: Serial-Section Electron Microscopy Image Registration Using a Spatial Transformer Network with Learned Features2017
- Registering large volume serial-section electron microscopy image sets for neural circuit reconstruction using FFT signal whitening2016
- Elastic volume reconstruction from series of ultra-thin microscopy sectionsNature Methods, 2012
- Volume electron microscopy for neuronal circuit reconstructionCurrent Opinion in Neurobiology, 2012
- Network anatomy and in vivo physiology of visual cortical neuronsNature, 2011
- Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopyJournal of Neuroscience Methods, 2010
- Uniform Serial Sectioning for Transmission Electron MicroscopyThe Journal of Neuroscience, 2006
- Object recognition from local scale-invariant features1999
- Multiresolution elastic matchingComputer Vision, Graphics, and Image Processing, 1989
- Pyramidal parametrics1983