Nucleus-Aware Self-Supervised Pretraining Using Unpaired Image-to-Image Translation for Histopathology Images
- 30 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 43 (1) , 459-472
Abstract
Self-supervised pretraining attempts to enhance model performance by obtaining effective features from unlabeled data, and has demonstrated its effectiveness in the field of histopathology images. Despite its success, few works concentrate on the extraction of nucleus-level information, which is essential for pathologic analysis. In this work, we propose a novel nucleus-aware self-supervised pretraining framework for histopathology images. The framework aims to capture the nuclear morphology and distribution information through unpaired image-to-image translation between histopathology images and pseudo mask images. The generation process is modulated by both conditional and stochastic style representations, ensuring the reality and diversity of the generated histopathology images for pretraining. Further, an instance segmentation guided strategy is employed to capture instance-level information. The experiments on 7 datasets show that the proposed pretraining method outperforms supervised ones on Kather classification, multiple instance learning, and 5 dense-prediction tasks with the transfer learning protocol, and yields superior results than other self-supervised approaches on 8 semi-supervised tasks. Our project is publicly available at https://github.com/zhiyuns/UNITPathSSL.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation in China (62022010)
- Beijing Natural Science Foundation-Haidian District Joint Fund in China (L222032)
- Beijing Hope Run Special Fund of Cancer Foundation of China (LC2018L02)
- Fundamental Research Funds for the Central Universities of China from the State Key Laboratory of Software Development Environment, Beihang University, China
- 111 Project in China (B13003)
- High Performance Computing (HPC) Resources with Beihang University
This publication has 44 references indexed in Scilit:
- Mask R-CNN2017
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- A Dataset and a Technique for Generalized Nuclear Segmentation for Computational PathologyIEEE Transactions on Medical Imaging, 2017
- Gland segmentation in colon histology images: The glas challenge contestMedical Image Analysis, 2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- A Stochastic Polygons Model for Glandular Structures in Colon Histology ImagesIEEE Transactions on Medical Imaging, 2015
- Histopathological Image Analysis: A ReviewIEEE Reviews in Biomedical Engineering, 2009
- Extracting and composing robust features with denoising autoencoders2008
- Biomarkers in Cancer Staging, Prognosis and Treatment SelectionNature Reviews Cancer, 2005