tUbe net: a generalisable deep learning tool for 3D vessel segmentation
Preprint
Abstract
Deep learning has become an invaluable tool for bioimage analysis but, while open-source cell annotation software such as cellpose are widely used, an equivalent tool for three-dimensional (3D) vascular annotation does not exist. With the vascular system being directly impacted by a broad range of diseases, there is significant medical interest in quantitative analysis for vascular imaging. We present a new deep learning model, coupled with a human-in-the-loop training approach, for segmentation of vasculature that is generalisable across tissues, modalities, scales and pathologies. To create a generalisable model, a 3D convolutional neural network was trained using curated data from modalities including optical imaging, computational tomography and photoacoustic imaging. Through this varied training set, the model was forced to learn common features of vessels cross-modality and scale. Following this, the pre-trained ‘foundation’ model was fine-tuned to different applications with a minimal amount of manually labelled ground truth data. It was found that the foundation model could be specialised to a new datasets using as little as 0.3% of the volume of said dataset for fine-tuning. The fine-tuned model was able to segment 3D vasculature with a high level of accuracy (DICE coefficient between 0.89 and 0.98) across a range of applications. These results show a general model trained on a highly varied data catalogue can be specialised to new applications with minimal human input. This model and training approach enables users to produce accurate segmentations of 3D vascular networks without the need to label large amounts of training data.Keywords
All Related Versions
- Published version: Biology Methods and Protocols, 10 (1), bpaf087.
This publication has 33 references indexed in Scilit:
- Retinal vessel segmentation via deep learning network and fully-connected conditional random fields2016
- Segmenting Retinal Blood Vessels WithDeep Neural NetworksIEEE Transactions on Medical Imaging, 2016
- Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?IEEE Transactions on Medical Imaging, 2016
- From generic to specific deep representations for visual recognition2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- Multi-Center MRI Carotid Plaque Component Segmentation Using Feature Normalization and Transfer LearningIEEE Transactions on Medical Imaging, 2014
- Hallmarks of Cancer: The Next GenerationCell, 2011
- Ridge-Based Vessel Segmentation in Color Images of the RetinaIEEE Transactions on Medical Imaging, 2004
- Advances in the Vascular Pathophysiology of Ischemic StrokeThrombosis Research, 2000
- Locating blood vessels in retinal images by piecewise threshold probing of a matched filter responseIEEE Transactions on Medical Imaging, 2000