MFF-SDD: A Bidirectional Guidance and Multiscale Multimodal Fusion Model for Small Defect Detection in Industrial Films
- 25 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 21 (12) , 9701-9711
Abstract
Detecting small defects in industrial biaxially oriented polypropylene (BOPP) films is challenging due to their limited visual cues, low contrast, and high similarity to background textures. Existing deep learning-based methods, which rely solely on visual data, often suffer from high miss rates and poor classification accuracy in such complex scenarios. To address these limitations, we propose the multimodal fusion-based film small defect detection (MFF-SDD) model, specifically designed for small defect detection in BOPP film production. Unlike conventional language-image models that neglect the bidirectional interaction between modalities, MFF-SDD introduces the bidirectional guidance multimodal cross-fusion module, which enhances visual–textual integration through mutual guidance and attention mechanisms, enabling more effective foreground focus and background suppression. Furthermore, the text and image cross-modality neighbors multiscale fusion module employs multiscale cross-modal fusion to preserve fine-grained details and exploit complementary features from both modalities, improving detection accuracy and reducing misclassification of small defects. We also present the film small defect (FSD) dataset, comprising 10 385 annotated bounding boxes across seven defect categories. Experimental results show that MFF-SDD surpasses state-of-the-art methods by 2.31% in average precision and 3.75% in recall on the FSD dataset and achieves leading performance on public benchmarks, such as PASCAL VOC and TinyPerson. These findings demonstrate the effectiveness and robustness of our approach in multimodal small defect detection. Our dataset and code will be made publicly available upon acceptance of this article for publication.Keywords
Funding Information
- Key Research and Development Program of Zhejiang Province (2024C03172)
- National Natural Science Foundation of China (62111530300)
- Natural Science Foundation of Zhejiang Province (Z24F020002)
This publication has 30 references indexed in Scilit:
- Predicting performance of object detection models in electron microscopy using random forestsDigital Discovery, 2025
- SuperSimpleNet: Unifying Unsupervised and Supervised Learning for Fast and Reliable Surface Defect Detection2024
- A survey of multimodal hybrid deep learning for computer vision: Architectures, applications, trends, and challengesInformation Fusion, 2023
- RGB oralscan video-based orthodontic treatment monitoringScience China Information Sciences, 2023
- Detecting Twenty-Thousand Classes Using Image-Level Supervision2022
- TinyDet: accurately detecting small objects within 1 GFLOPsScience China Information Sciences, 2022
- Open Vocabulary Object Detection with Pseudo Bounding-Box Labels2022
- Multimodal research in vision and language: A review of current and emerging trendsInformation Fusion, 2022
- TIDE: A General Toolbox for Identifying Object Detection Errors2020
- End-to-End Object Detection with Transformers2020