Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification
Preprint
Abstract
Masked autoencoders (MAEs) have displayed significant potential in the classification and semantic segmentation of medical images in the last year. Due to the high similarity of human tissues, even slight changes in medical images may represent diseased tissues, necessitating fine-grained inspection to pinpoint diseased tissues. The random masking strategy of MAEs is likely to result in areas of lesions being overlooked by the model. At the same time, inconsistencies between the pre-training and fine-tuning phases impede the performance and efficiency of MAE in medical image classification. To address these issues, we propose a medical supervised masked autoencoder (MSMAE) in this paper. In the pre-training phase, MSMAE precisely masks medical images via the attention maps obtained from supervised training, contributing to the representation learning of human tissue in the lesion area. During the fine-tuning phase, MSMAE is also driven by attention to the accurate masking of medical images. This improves the computational efficiency of the MSMAE while increasing the difficulty of fine-tuning, which indirectly improves the quality of MSMAE medical diagnosis. Extensive experiments demonstrate that MSMAE achieves state-of-the-art performance in case with three official medical datasets for various diseases. Meanwhile, transfer learning for MSMAE also demonstrates the great potential of our approach for medical semantic segmentation tasks. Moreover, the MSMAE accelerates the inference time in the fine-tuning phase by 11.2% and reduces the number of floating-point operations (FLOPs) by 74.08% compared to a traditional MAE.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Improving Masked Autoencoders by Learning Where to Mask2023
- DCELANM‐Net : Medical image segmentation based on dual channel efficient layer aggregation network with learnerInternational Journal of Imaging Systems and Technology, 2023
- EDMAE: An efficient decoupled masked autoencoder for standard view identification in pediatric echocardiographyBiomedical Signal Processing and Control, 2023
- Self-Supervision for Medical Image Classification: State-of-the-Art Performance with ~100 Labeled Training Samples per ClassBioengineering, 2023
- Good Helper Is around You: Attention-Driven Masked Image ModelingProceedings of the AAAI Conference on Artificial Intelligence, 2023
- Hard Patches Mining for Masked Image Modeling2023
- Brain tumor detection and classification using machine learning: a comprehensive surveyComplex & Intelligent Systems, 2021
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesionsScientific Data, 2018
- Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep LearningInvestigative Ophthalmology & Visual Science, 2016