Faming Shao

Few-shot object detection (FSOD) aims to retain the performance of detector when only given scarce annotated instances. We reckon that its difficulty lies in the fact that the scare positive samples restrict the accurate construction of the eigenspace of involved categories. In this paper, we proposed a novel FSOD detector based on refining the eigenspace, which is implemented through a pure positive augmentation, a full feature mining module and a modified loss function. The pure positive augmentation expands the quantity and enriches the scale distribution of positive samples, inhibiting the expansion of negative samples. The full feature mining module enables the model to mining more information about objects. The modified loss function drives prediction results closer to ground truths. We apply these two improvements to YOLOv4, the representative of one-stage detector, which is termed YOLOv4-FS. On PASCAL VOC and MS COCO datasets, our YOLOv4-FS achieves competitive performance compared with existing progressive detectors.

Military vehicle object detection technology in complex environments is the basis for the implementation of reconnaissance and tracking tasks for weapons and equipment, and is of great significance for information and intelligent combat. In response to the poor performance of traditional detection algorithms in military vehicle detection, we propose a military vehicle detection method based on hierarchical feature representation and reinforcement learning refinement localization, referred to as MVODM. First, for the military vehicle detection task, we construct a reliable dataset MVD. Second, we design two strategies, hierarchical feature representation and reinforcement learning-based refinement localization, to improve the detector. The hierarchical feature representation strategy can help the detector select the feature representation layer suitable for the object scale, and the reinforcement learning-based refinement localization strategy can improve the accuracy of the object localization boxes. The combination of these two strategies can effectively improve the performance of the detector. Finally, the experimental results on the homemade dataset show that our proposed MVODM has excellent detection performance and can better accomplish the detection task of military vehicles.

Underwater images have low quality, and underwater targets have different sizes. The mainstream target detection networks cannot achieve good results in detecting objects from underwater images. In this study, a lightweight underwater multiscale target detection model with an attention mechanism is designed to solve the above problems. In this model, MobileNetv3 is used as the backbone network for preliminary feature extraction. The lightweight feature extraction module (LFEM) pays attention to the feature map at the channel and space levels. The features with large weights are promoted, while the features with small weights are suppressed. Meanwhile, cross-group information exchange enriches the semantic information and location information of the objects. The context aggregation module (CIAM) pools the extracted feature maps to obtain feature pyramids, and it uses the upsampling-feature refinement-cascade addition (URC) method to effectively fuse global context information and enhance the feature representation. The scale normalization for feature pyramids (SNFP) performs adaptive multiscale perception and multianchor detection on feature maps to cover objects of different sizes and realize multiscale object detection in underwater images. The proposed network can realize lightweight feature extraction, effectively handle the global relationship between the underwater scene and the object while expanding the receptive field, traverse the objects of different scales, and achieve adaptive multianchor detection of multiscale objects in underwater images. The experimental results indicate that our method achieves an average accuracy of 81.94% and a detection speed of 44.3 FPS on a composite dataset. Also, our method is better than the mainstream object detection networks in terms of detection accuracy, lightweight design, and real-time performance.

Detecting objects in aerial images is a challenging task due to the large-scale variations and arbitrary orientations with tiny instances. A new multi-scale transformer-based aerial objects detector called MStrans is proposed in this paper to deal with the challenges in aerial detection. To detect remote instances, MStrans adopts a multi-scale patch embedding transformer (MViT) to extract the global features of the image effectively. Furthermore, to capture the different discriminant features for classification and regression branch tasks, the partial interactive fusion module (PIFM) is designed to enhance the semantic expression of the key features of classification and regression tasks by using the strategy of interactive modeling of adjacent layer features. In addition, considering that the transformer may worsen the local feature details while capturing long-distance feature dependencies, this paper designs a global to local interactive fusion module (GLIFM). It uses the advantage of convolution to extract local features to enrich the detailed information in the transformer. Experiments were carried out on DOTA and DIOR datasets, and the MStrans achieves superior detection performances compared with other approaches.

Multispectral pedestrian detection based on deep learning can provide a robust and accurate detection under different illumination conditions, which has important research significance in safety. In order to reduce the log-average miss rate of the object under different illumination conditions, a new one-stage detector suitable for multispectral pedestrian detection is proposed. First, in order to realize the complementarity between the information flows of the two modalities in the feature extraction stage to reduce the object loss, a low-cost cross-modality feature complementary module (CFCM) is proposed. Second, in order to suppress the background noise in different environments and enhance the semantic information and location information of the object, so as to reduce the error detection of the object, an attention-based feature enhancement fusion module (AFEFM) is proposed. Thirdly, through the feature complementarity of color-thermal image pair and the multi-scale fusion of depth feature layer, the horizontal and vertical multi-dimensional data mining of parallel deep neural network is realized, which provides effective data support for object detection algorithm. Finally, through the reasonable arrangement of proposed modules, a robust multispectral detection framework is proposed. The experimental results on the Korea Advanced Institute of Science and Technology (KAIST) pedestrian benchmark show that the proposed method has the lowest log-average miss rate compared with other state-of-the-art multispectral pedestrian detectors, and has a good balance in speed and accuracy.

Existing video captioning methods usually ignore the important fine-grained semantic attributes, the video diversity, as well as the association and motion state between objects within and between frames. Thus, they cannot adapt to small sample data sets. To solve the above problems, this paper proposes a novel video captioning model and an adversarial reinforcement learning strategy. Firstly, an object-scene relational graph model is designed based on the object detector and scene segmenter to express the association features. The graph is encoded by the graph neural network to enrich the expression of visual features. Meanwhile, a trajectory-based feature representation model is designed to replace the previous data-driven method to extract motion and attribute information, so as to analyze the object motion in the time domain and establish the connection between the visual content and language under small data sets. Finally, an adversarial reinforcement learning strategy and a multi- branch discriminator are designed to learn the relationship between the visual content and corresponding words so that rich language knowledge is integrated into the model. Experimental results on three standard datasets and one small sample dataset indicate that our proposed method achieves state-of-the-art performance. Also, the ablation experiments and visualization results verify the effectiveness of proposed each strategy.

The remote sensing images in large scenes have a complex background, and the types, sizes, and postures of the targets are different, making object detection in remote sensing images difficult. To solve this problem, an end-to-end multi-size object detection method based on a dual attention mechanism is proposed in this paper. First, the MobileNets backbone network is used to extract multi-layer features of remote sensing images as the input of MFCA, a multi-size feature concentration attention module. MFCA employs an attention mechanism to suppress noise, enhance effective feature reuse, and improve the adaptability of the network to multi-size target features through multi-layer convolution operation. Then, TSDFF (two-stage deep feature fusion module)deeply fuses the feature maps output by MFCA to maximize the correlation between the feature sets and especially improve the feature expression of small targets. Next, the GLCNet (global-local context network) and the SSA (significant simple attention module) distinguish the fused features and screen out useful channel information, which makes the detected features more representative. Finally, the loss function is improved to truly reflect the difference between the candidate frames and the real frames, enhancing the network’s ability to predict complex samples. The performance of our proposed method is compared with other advanced algorithms on NWPU VHR-10, DOTA, RSOD open datasets. Experimental results show that our proposed method achieves the best AP (average precision) and mAP (mean average precision), indicating that the method can accurately detect multi-type, multi-size, and multi-posture targets with high adaptability.

Bridge crack detection is essential to ensure bridge safety. The introduction of deep learning technology has made it possible to detect bridge cracks automatically and accurately. In this study, the Inception-Resnet-v2 algorithm was systematically improved and applied to the real-time detection of bridge cracks. We propose an end-to-end bridge crack detection model based on a convolutional neural network. This model combines the advantages of Inception convolution and residual networks, broadening the network width and alleviating the training problem of the deep network. The calculation speed is improved while still ensuring accuracy. Multi-scale feature fusion enables the network to extract contextual information of different scales, which improves the accuracy of crack recognition. The GKA (K-means clustering method based on a genetic algorithm) realizes the accurate segmentation of the target area, greatly enhances the clustering effect, and effectively improves the detection speed. In this model, large fracture datasets are used for training and testing without pre-training. The experimental results show that the performance of this method was improved in all aspects: accuracy, 99.24%; recall, 99.03%; F-measure, 98.79%; and FPS(Frames Per Second), 196.

The hydraulic pump plays a very important role in the safe and stable operation of the hydraulic system. Once it fails, it will cause immeasurable losses to the entire hydraulic system. But in practice, because hydraulic pump often works under strong noise background, the fault characteristics of its vibration signals are often very weak and difficult to extract. To solve this problem, this paper proposes an effective time series dynamic feature extraction method, which is based on the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and composite multi-scale basic scale entropy (CMBSE). On this basis, a new hydraulic pump fault diagnosis method is proposed by combining t-distributed stochastic neighbor embedding (t-SNE) and whale optimization algorithm kernel extreme learning machine (WOA-KELM). First, CEEMDAN is used to decompose the fault signals of the hydraulic pump, and CMBSE is used to quantify the decomposed IMF components to obtain the fault characteristics of the different states of the hydraulic pump, and then use t-SNE to visualize the dimensionality reduction, and finally input into the WOA-KELM-based fault classifier for state identification. The experimental results show that this method can effectively extract the weak signal features under strong noise background, and has broad application prospects for hydraulic pump fault diagnosis.