CheSS: Chest X-Ray Pre-trained Model via Self-supervised Contrastive Learning
Authors:
Open Access
- 26 January 2023
- journal article
- research article
- Published by Springer Nature in Journal of Digital Imaging
- Vol. 36 (3) , 902-910
Abstract
Training deep learning models on medical images heavily depends on experts’ expensive and laborious manual labels. In addition, these images, labels, and even models themselves are not widely publicly accessible and suffer from various kinds of bias and imbalances. In this paper, chest X-ray pre-trained model via self-supervised contrastive learning (CheSS) was proposed to learn models with various representations in chest radiographs (CXRs). Our contribution is a publicly accessible pretrained model trained with a 4.8-M CXR dataset using self-supervised learning with a contrastive learning and its validation with various kinds of downstream tasks including classification on the 6-class diseases in internal dataset, diseases classification in CheXpert, bone suppression, and nodule generation. When compared to a scratch model, on the 6-class classification test dataset, we achieved 28.5% increase in accuracy. On the CheXpert dataset, we achieved 1.3% increase in mean area under the receiver operating characteristic curve on the full dataset and 11.4% increase only using 1% data in stress test manner. On bone suppression with perceptual loss, we achieved improvement in peak signal to noise ratio from 34.99 to 37.77, structural similarity index measure from 0.976 to 0.977, and root-square-mean error from 4.410 to 3.301 when compared to ImageNet pretrained model. Finally, on nodule generation, we achieved improvement in Fréchet inception distance from 24.06 to 17.07. Our study showed the decent transferability of CheSS weights. CheSS weights can help researchers overcome data imbalance, data shortage, and inaccessibility of medical image datasets. CheSS weight is available at https://github.com/mi2rl/CheSS.Keywords
Funding Information
- Korea Health Industry Development Institute (HI18C2383)
This publication has 30 references indexed in Scilit:
- Occurrence of the potent mutagens 2- nitrobenzanthrone and 3-nitrobenzanthrone in fine airborne particlesScientific Reports, 2019
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- Introduction and ‘Checklist’2017
- ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases2017
- Deep Residual Learning for Image Recognition2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Unpatients—why patients should own their medical dataNature Biotechnology, 2015
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- HIPAA Regulations — A New Era of Medical-Record Privacy?New England Journal of Medicine, 2003
- Glossary of terms for thoracic radiology: recommendations of the Nomenclature Committee of the Fleischner SocietyAmerican Journal of Roentgenology, 1984