Yulong Liu

Printed circuit board (PCB) assembly, on the basis of surface mount electronic component welding, is one of the most important electronic assembly processes, and its defect detection is also an important part of industrial generation. The traditional two-stage target detection algorithm model has a large number of parameters and the runtime is too long. The single-stage target detection algorithm has a faster running time, but the detection accuracy needs to be improved. To solve this problem, we innovated and modified the YOLO11n model. Firstly, we used the Retention Block (RetBlock) to improve the C3K2 module in the backbone, creating the RetC3K2 module, which makes up for the limitation of the original module’s limited, purely convolutional local receptive field. Secondly, the neck structure of the original model network is fused with a Multi-Branch Auxiliary Feature Pyramid Network (MAFPN) structure and turned into a multi-branch auxiliary neck network, which enhances the model’s ability to fuse multiple scaled characteristics and conveys diverse information about the gradient for the output layer. The improved YOLO11n model improves its mAP50 by 0.023 (2.5%) and mAP75 by 0.026 (2.8%) in comparison with the primitive model network, and detection precision is significantly improved, proving the superiority of our proposed approach.

Soldering of printed circuit board (PCB)-based surface-mounted assemblies is a critical process, and to enhance the accuracy of detecting their multi-targeted soldering defects, we propose an automated sample generation method that combines ControlNet and a Stable Diffusion Model. This method can expand the dataset by quickly obtaining sample images with high quality containing both defects and normal detection targets. Meanwhile, we propose the Cascade Mask R-CNN model with ConvNeXt as the backbone, which performs well in dealing with multi-target defect detection tasks. Unlike previous detection methods that can only detect a single component, it can detect all components in the region. The results of the experiment demonstrate that the detection accuracy of our proposed approach is significantly enhanced over the previous convolutional neural network model, with an increase of more than 10.5% in the mean accuracy precision (mAP) and 9.5% in the average recall (AR).

In order to improve the efficiency and accuracy of multitarget detection of soldering defects on surface-mounted components in Printed Circuit Board (PCB) fabrication, we propose a sample generation method using Stable Diffusion Model and ControlNet, as well as a defect detection method based on the Swin Transformer. The method consists of two stages: First, high-definition original images collected in industrial production and the corresponding prompts are input to Stable Diffusion Model and ControlNet for automatic generation of nonindependent samples. Subsequently, we integrate Swin Transformer as the backbone into the Cascade Mask R-CNN to improve the quality of defect features extracted from the samples for accurate detection box localization and segmentation. Instead of segmenting individual components on the PCB, the method inspects all components in the field of view simultaneously over a larger area. The experimental results demonstrate the effectiveness of our method in scaling up nonindependent sample datasets, thereby enabling the generation of high-quality datasets. The method accurately recognizes targets and detects defect types when performing multitarget inspection on printed circuit boards. The analysis against other models shows that our improved defect detection and segmentation method improves the Average Recall (AR) by 2.8% and the mean Average Precision (mAP) by 1.9%.

In order to adapt to the high standard of defect detection in industrial production for electronic component soldering, and to address the issues of unbalanced integrated circuit (IC) defect samples and the difficulty of defect detection and segmentation, this paper proposes a model based on the Swin Transformer architecture for automatic detection and segmentation of solder defects in electronic components. The approach involves feeding real defect samples collected from industrial production into the Cycle-Generative Adversarial Network, which generates a balanced and sufficient sample dataset through adversarial generation. The Swin Transformer framework is then applied to the current mainstream Mask R-CNN, resulting in an improved network model named Swin Transformer Based Mask R-CNN (ST-mask-rcnn). This model effectively detects IC pin soldering defects and solder joints of surface mounted devices (SMD). The experimental results demonstrate that the proposed method successfully addresses the sample imbalance problem in electronic components. It achieves a 6% and 18% improvement in mean average precision for IC pin soldering defects and SMD resistive solder joints detection, respectively. Additionally, it enhances the segmentation accuracy to 99.81% and 99.92%, with improved detection speed.

When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to imbalances in sample datasets and the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network (GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to automatically generate masks for the defect images. Experimental results demonstrate the effectiveness of our proposed method, as it can simultaneously generate liquid crystal image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.

When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to the imbalance in samples dataset, as well as the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network(GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to generate masks for the defect images automatically. Experimental results shows the effectiveness of our proposed method, as it allows for the simultaneous generation of LCD image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.

Surface defect inspection based on machine vision is often affected by uneven illumination. In order to improve the inspection rate of surface defects inspection under uneven illumination condition, this paper proposes a method for detecting surface image defects based on convolutional neural network, which is based on the adjustment of convolutional neural networks, training parameters, changing the structure of the network, to achieve the purpose of accurately identifying various defects. Experimental on defect inspection of copper strip and steel images shows that the convolutional neural network can automatically learn features without preprocessing the image, and correct identification of various types of image defects affected by uneven illumination, thus overcoming the drawbacks of traditional machine vision inspection methods under uneven illumination.

The Mask R-CNN-based object detection method is typically very time-consuming and laborious since it involves obtaining the required target object masks during training. Therefore, in order to automatically generate the image mask, we propose a GrabCut-based automated mask generation method for object detection. The proposed method consists of two stages. The first stage is based on GrabCut’s interactive image segmentation method to generate the mask. The second stage is based on the object detection network of Mask R-CNN, which uses the mask from the previous stage together with the original input image and the associated label information for training. The Mask R-CNN model then automatically detects the relevant objects during testing. During experimentation with three objects from the Berkeley Instance Recognition Dataset, this method achieved a mean of average precision (mAP) value of over 95% for segmentation. The proposed method is simple and highly efficient in obtaining the mask of a segmented target object.

When the deep learning method is applied to the detection of microscopic defects with uneven brightness on the low-contrast LCD surface, due to the imbalance between the positive samples and negative samples when training the deep learning method, we propose a method to automatically generate samples based on a deep generative network model. First, a small number of defect samples are generated through a generative adversarial network to generate an expanded sample data set, and then the samples in these data sets are manually marked to highlight the defect parts, and the mask of the defect image is obtained for Mask R-CNN detection. The experimental results confirm that LCD image samples can be automatically generated by the proposed method, and the experiments on Mask R-CNN also confirm the effectiveness of our proposed method.

In this paper, we address the semantic segmentation problem using single-layer networks. This network is used for unsupervised feature learning for the available RGB image and the depth image. A significant contribution of the proposed approach is that the dictionary is selected from the existing samples using the L _{2, 1} optimization. Such a dictionary can capture more meaningful representative samples and exploit intrinsic correlation between features from different modalities. The experimental results on the public NYU dataset show that this strategy dramatically improves the classification performance, compared with existing dictionary learning approach. In addition, we perform experimental verification using the practical robot platforms and show promising results.