Few-Shot Photovoltaic Film Defect Detection With Contextual Ensemble Language-Image Multimodal Network
- 13 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 21 (8) , 6274-6283
Abstract
The automatic detection of defects in industrial photovoltaic film is crucial for ensuring the reliability of photovoltaic modules. Key challenges include limited defect samples, interclass feature similarities, and interference from complex backgrounds. Existing deep learning-based methods require large-scale datasets and focus solely on visual data, which limits their effectiveness in few-shot defect detection (FSDD). To address these challenges, we propose Contextual Ensemble Language-Image multimodal Network (CELIN), which enhances FSDD in photovoltaic films by incorporating textual information through prompt tuning. Unlike traditional language-image models that rely on single fixed text prompts, CELIN employs a position-aware context ensemble strategy to integrate position-specific prompt vectors, enabling the model to capture global contextual information and reduce background interference. In addition, a cross class mask method is introduced to differentiate between similar defect categories by blocking interclass interactions during attention computation, thereby minimizing misclassification. Extensive experiments on our own few-shot photovoltaic film defect dataset and various public benchmarks demonstrates that CELIN significantly outperforms existing methods.Keywords
Funding Information
- Key Research and Development Program of Zhejiang Province (2024C03172)
This publication has 32 references indexed in Scilit:
- Few-shot object detection: Research advances and challengesInformation Fusion, 2024
- SPot-the-Difference Self-supervised Pre-training for Anomaly Detection and Segmentation2022
- Learning to Prompt for Vision-Language ModelsInternational Journal of Computer Vision, 2022
- Spatial reasoning for few-shot object detectionPattern Recognition, 2021
- TIDE: A General Toolbox for Identifying Object Detection Errors2020
- Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector2020
- MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection2019
- LVIS: A Dataset for Large Vocabulary Instance Segmentation2019
- Reconstructing PASCAL VOC2014
- Microsoft COCO: Common Objects in Context2014