Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-supervision
- 14 October 2023
- book chapter
- Published by Springer Nature
- Vol. 14293, 94-104
Abstract
Human anatomy is the foundation of medical imaging and boasts one striking characteristic: its hierarchy in nature, exhibiting two intrinsic properties: (1) locality: each anatomical structure is morphologically distinct from the others; and (2) compositionality: each anatomical structure is an integrated part of a larger whole. We envision a foundation model for medical imaging that is consciously and purposefully developed upon this foundation to gain the capability of “understanding” human anatomy and to possess the fundamental properties of medical imaging. As our first step in realizing this vision towards foundation models in medical imaging, we devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy. Our extensive experiments demonstrate that the SSL pretrained model, derived from our training strategy, not only outperforms state-of-the-art (SOTA) fully/self-supervised baselines but also enhances annotation efficiency, offering potential few-shot segmentation capabilities with performance improvements ranging from 9% to 30% for segmentation tasks compared to SSL baselines. This performance is attributed to the significance of anatomy comprehension via our learning strategy, which encapsulates the intrinsic attributes of anatomical structures—locality and compositionality—within the embedding space, yet overlooked in existing SSL methods. All code and pretrained models are available at GitHub.com/JLiangLab/Eden.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Guest Editorial Annotation-Efficient Deep Learning: The Holy Grail of Medical ImagingIEEE Transactions on Medical Imaging, 2021
- Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-Supervised LearningIEEE Transactions on Medical Imaging, 2021
- A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis2021
- A Structure-Aware Relation Network for Thoracic Diseases Detection and SegmentationIEEE Transactions on Medical Imaging, 2021
- Emergent linguistic structure in artificial neural networks trained by self-supervisionProceedings of the National Academy of Sciences, 2020
- Progressive decomposition: a method of coarse-to-fine image parsing using stacked networksMultimedia Tools and Applications, 2020
- ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases2017
- Robust Vessel Segmentation in Fundus ImagesInternational Journal of Biomedical Imaging, 2013
- EyePACS: An Adaptable Telemedicine System for Diabetic Retinopathy ScreeningJournal of Diabetes Science and Technology, 2009
- Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public databaseMedical Image Analysis, 2006