Tianzhao Zhang

Few-shot object detection (FSOD) is proposed to solve the application problem of traditional detectors in scenarios lacking training samples. The meta-learning methods have attracted the researchers’ attention for their excellent generalization performance. They usually select the same class of support features according to the query labels to weight the query features. However, the model cannot possess the ability of active identification only by using the same category support features, and feature selection causes difficulties in the testing process without labels. The single-scale feature of the model also leads to poor performance in small object detection. In addition, the hard samples in the support branch impact the backbone’s representation of the support features, thus impacting the feature weighting process. To overcome these problems, we propose a multi-scale feature fusion and attentive learning (MSFFAL) framework for few-shot object detection. We first design the backbone with multi-scale feature fusion and channel attention mechanism to improve the model’s detection accuracy on small objects and the representation of hard support samples. Based on this, we propose an attention loss to replace the feature weighting module. The loss allows the model to consistently represent the objects of the same category in the two branches and realizes the active recognition of the model. The model no longer depends on query labels to select features when testing, optimizing the model testing process. The experiments show that MSFFAL outperforms the state-of-the-art (SOTA) by 0.7–7.8% on the Pascal VOC and exhibits 1.61 times the result of the baseline model in MS COCO’s small objects detection.

The emergence of Transformer has led to the rapid development of video understanding, but it also brings the problem of high computational complexity. Previously, there were methods to divide the feature maps into windows along the spatiotemporal dimensions and then calculate the attention. There are also methods to perform down-sampling during attention computation to reduce the spatiotemporal resolution of features. Although the complexity is effectively reduced, there is still room for further optimization. Thus, we present the Windows and Linear Transformer (WLiT) for efficient video action recognition, by combining Spatial-Windows attention with Linear attention. We first divide the feature maps into multiple windows along the spatial dimensions and calculate the attention separately inside the windows. Therefore, our model further reduces the computational complexity compared with previous methods. However, the perceptual field of Spatial-Windows attention is small, and global spatiotemporal information cannot be obtained. To address this problem, we then calculate Linear attention along the channel dimension so that the model can capture complete spatiotemporal information. Our method achieves better recognition accuracy with less computational complexity through this mechanism. We conduct extensive experiments on four public datasets, namely Something-Something V2 (SSV2), Kinetics400 (K400), UCF101, and HMDB51. On the SSV2 dataset, our method reduces the computational complexity by 28% and improves the recognition accuracy by 1.6% compared to the State-Of-The-Art (SOTA) method. On the K400 and two other datasets, our method achieves SOTA-level accuracy while reducing the complexity by about 49%.

The Non-local self-attention mechanism can significantly improve the capability of feature representation with long-range dependencies at the cost of high computational complexity. To address the issue, the self-attention-based autoregressive axial transformer has been proposed to apply attention along a single axis of the feature maps instead of the whole ones with large receptive fields. It performs axial-attention twice along the height- and width-axis respectively in the spatial dimension of the feature maps for the image data. However, there is still room for improvement. We can convert the 2D spatial feature map into a 1D feature sequence and just perform axial-attention once along it to save more computing resources. Motivated by the insight, we propose an Efficient Axial-Attention Network (EAAN) for video-based person re-identification (Re-ID) to reduce computation and improve accuracy by serializing feature maps with multi-granularity and reducing the number of axial-attention runs. We also introduce a deserialization approach that can restore the shape of the feature maps. Moreover, we expand the CTN (Channel Transformer Network) to a wider range of uses. Additionally, we verify that the serialized feature sequence can enhance expressiveness in our EAAN with lower complexity. Experiments on MARS and DukeMTMC-VideoReID (DukeV) datasets show outstanding performance in computation efficiency and accuracy. It not only outperforms the state-of-the-art method on MARS by 0.3% in both Rank-1 and mAP, and surpasses that on DukeV by 0.1% in Rank-1 with equal mAP, but also reduces parameters and GFLOPS (Giga Floating-point Operations Per Second) by 16.9% and 6.6% respectively compared to another axial-attention-based method. The code will be available at https://github.com/hopstone/EAAN.