ReSSL: Boosting Semi-Supervised Learning with Reconstruction-Aware Masked Autoencoders
Authors:
- Wenjin ZhangWZWenjin Zhang
- Sen YangSYSen Yang
- Syed Muhammad AnwarSASyed Muhammad Anwar
- Chenyang GaoCGChenyang Gao
- Wanzhao YangWYWanzhao Yang
- Mary S. KimMKMary S. Kim
- Aleksandra SarcevicASAleksandra Sarcevic
- Marius George LinguraruMLMarius George Linguraru
- Randall S. BurdRBRandall S. Burd
- Ivan MarsicIMIvan Marsic
Preprint
- 11 February 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Foundation vision models show remarkable performance across various tasks but often require domain-specific adaptation to achieve optimal results. However, fine-tuning these models is challenging due to the high cost of obtaining labeled data, particularly in specialized domains like medical imaging. Semi-supervised learning (SSL) addresses this challenge by leveraging limited labeled data to generate pseudo-labels for unlabeled data. Existing SSL methods rely on confidence-based thresholding, which biases training towards high-confidence samples while neglecting uncertain ones, leading to suboptimal generalization.To overcome this limitation, we propose \algo, an SSL booster that enhances representation learning by incorporating Masked Autoencoder (MAE)-based reconstruction loss and Supplementary MixUp (SMU). Unlike traditional MAE-based pretraining followed by fine-tuning, our method jointly optimizes reconstruction and classification, learning from all data and ensuring feature alignment with the target task (i.e., classification). To avoid overfitting issues of MixUp in SSLs, we propose SMU that balances original and MixUp-augmented data. This approach also addressed the challenge of determining the optimal transition point for augmentation.Our approach achieves state-of-the-art performance on multiple SSL benchmarks, including 97.5\% accuracy on Euro-SAT with 2 labeled samples per class and 74.6\% accuracy on R2PPE recognition with 20 labeled samples per class. These results show that \algo effectively improves SSL by leveraging all available unlabeled data, reducing reliance on confidence thresholds, and enhancing generalization through joint optimization of MAE and classification objectives.Keywords
This publication has 11 references indexed in Scilit:
- EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
- Introducing Eurosat: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification2018
- Learning Features by Watching Objects Move2017
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Colorful Image Colorization2016
- Context Encoders: Feature Learning by Inpainting2016
- Unsupervised Visual Representation Learning by Context Prediction2015
- Unsupervised Learning of Visual Representations Using Videos2015
- ImageNet: A large-scale hierarchical image database2009
- Self-organizing neural network that discovers surfaces in random-dot stereogramsNature, 1992