Optimization of vision transformer-based detection of lung diseases from chest X-ray images
Authors:
Open Access
- 8 July 2024
- journal article
- research article
- Published by Springer Nature in BMC Medical Informatics and Decision Making
- Vol. 24 (1) , 1-8
Abstract
Background: Recent advances in Vision Transformer (ViT)-based deep learning have significantly improved the accuracy of lung disease prediction from chest X-ray images. However, limited research exists on comparing the effectiveness of different optimizers for lung disease prediction within ViT models. This study aims to systematically evaluate and compare the performance of various optimization methods for ViT-based models in predicting lung diseases from chest X-ray images. Methods: This study utilized a chest X-ray image dataset comprising 19,003 images containing both normal cases and six lung diseases: COVID-19, Viral Pneumonia, Bacterial Pneumonia, Middle East Respiratory Syndrome (MERS), Severe Acute Respiratory Syndrome (SARS), and Tuberculosis. Each ViT model (ViT, FastViT, and CrossViT) was individually trained with each optimization method (Adam, AdamW, NAdam, RAdam, SGDW, and Momentum) to assess their performance in lung disease prediction. Results: When tested with ViT on the dataset with balanced-sample sized classes, RAdam demonstrated superior accuracy compared to other optimizers, achieving 95.87%. In the dataset with imbalanced sample size, FastViT with NAdam achieved the best performance with an accuracy of 97.63%. Conclusions: We provide comprehensive optimization strategies for developing ViT-based model architectures, which can enhance the performance of these models for lung disease prediction from chest X-ray images.Keywords
All Related Versions
Funding Information
- The National Research Foundation of Korea (NRF) funded by the Korean government (2019R1A5A2026045)
- the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare, Republic of Korea (HR21C1003)
This publication has 28 references indexed in Scilit:
- High-precision multiclass classification of lung disease through customized MobileNetV2 from chest X-ray imagesComputers in Biology and Medicine, 2023
- IEViT: An enhanced vision transformer architecture for chest X-ray image classificationComputer Methods and Programs in Biomedicine, 2022
- xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using RadiographyIEEE Journal of Translational Engineering in Health and Medicine, 2021
- A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future PromisesProceedings of the IEEE, 2021
- The effect of batch size on the generalizability of the convolutional neural networks on a histopathology datasetICT Express, 2020
- COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray imagesScientific Reports, 2020
- CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray imagesComputer Methods and Programs in Biomedicine, 2020
- COVID-19 identification in chest X-ray images on flat and hierarchical classification scenariosComputer Methods and Programs in Biomedicine, 2020
- ImageNet: A large-scale hierarchical image database2009
- An improved algorithm for neural network classification of imbalanced training setsIEEE Transactions on Neural Networks, 1993