Evaluating diagnostic content of AI-generated chest radiography: A multi-center visual Turing test
Open Access
- 12 April 2023
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 18 (4) , e0279349
Abstract
Background: Accurate interpretation of chest radiographs requires years of medical training, and many countries face a shortage of medical professionals to meet such requirements. Recent advancements in artificial intelligence (AI) have aided diagnoses; however, their performance is often limited due to data imbalance. The aim of this study was to augment imbalanced medical data using generative adversarial networks (GANs) and evaluate the clinical quality of the generated images via a multi-center visual Turing test. Methods: Using six chest radiograph datasets, (MIMIC, CheXPert, CXR8, JSRT, VBD, and OpenI), starGAN v2 generated chest radiographs with specific pathologies. Five board-certified radiologists from three university hospitals, each with at least five years of clinical experience, evaluated the image quality through a visual Turing test. Further evaluations were performed to investigate whether GAN augmentation enhanced the convolutional neural network (CNN) classifier performances. Results: In terms of identifying GAN images as artificial, there was no significant difference in the sensitivity between radiologists and random guessing (result of radiologists: 147/275 (53.5%) vs result of random guessing: 137.5/275, (50%); p = .284). GAN augmentation enhanced CNN classifier performance by 11.7%. Conclusion: Radiologists effectively classified chest pathologies with synthesized radiographs, suggesting that the images contained adequate clinical information. Furthermore, GAN augmentation enhanced CNN performance, providing a bypass to overcome data imbalance in medical AI training. CNN based methods rely on the amount and quality of training data; the present study showed that GAN augmentation could effectively augment training data for medical AI.Keywords
Funding Information
- MD-PhD/Medical Scientist Training Program, Korea Health Industry Development Institute, Ministry of Health and Welfare
- National Research Foundation of Korea, Ministry of ICT, Science, and Technology (No-2021R1C1C2095529)
This publication has 29 references indexed in Scilit:
- Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologistsPLOS Medicine, 2018
- Sensitivity of chest X-ray for lung cancer: systematic reviewBritish Journal of General Practice, 2018
- Chest X-Ray Analysis of Tuberculosis by Deep Learning with Segmentation and Augmentation2018
- Generative Adversarial Networks: An OverviewIEEE Signal Processing Magazine, 2018
- A survey on deep learning in medical image analysisMedical Image Analysis, 2017
- Generative adversarial networks: introduction and outlookIEEE/CAA Journal of Automatica Sinica, 2017
- ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases2017
- Preparing a collection of radiology examinations for distribution and retrievalJournal of the American Medical Informatics Association, 2015
- Sensitivity and specificity of chest x-ray screening for lung cancerCancer, 2000
- Development of a Digital Image Database for Chest Radiographs With and Without a Lung NoduleAmerican Journal of Roentgenology, 2000