Qing Li

目的人脸表情识别（Facial Expression Recognition， FER）是人机交互与情感计算领域的核心技术，在智能监控、教育评估和医疗诊断等方面具有广阔的应用前景。近年来，Vision Transformer（ViT）架构凭借其强大的建模能力以及与多模态大模型良好的兼容性，正逐步取代传统卷积神经网络，成为FER领域的研究热点。然而，现有基于ViT的FER方法在预训练与后训练策略的选择上缺乏系统性探究，不同方法间性能差异显著，且原生ViT模型通常难以直接取得优于CNN或混合架构的识别效果。方法针对上述问题，本文首先系统梳理并实验对比了监督预训练、自监督预训练及多种后训练策略在原生ViT上的适用性。进一步，结合人脸表情识别需捕捉面部细微形变的特性，本文提出一种混合监督对比学习后训练方法，通过在监督对比学习中引入样本混合策略，有效扩展正样本多样性，增强模型对细粒度特征的判别能力。基于系统评估，本文提出掩码图像预训练与混合监督对比学习后训练相结合的原生ViT训练范式MIMIC（Masked Image Pre-training with Mixed Contrastive Learning）。结果在RAF-DB、AffectNet和FERPlus三个基准数据集上的大量实验表明，MIMIC在不引入任何卷积模块或复杂辅助模块的条件下，显著优于现有训练范式，展现出更强的特征学习能力，并随着模型规模增大，该方法未出现明显性能饱和现象，结论MIMIC能够有效提升原生ViT在面部表情识别任务中的特征学习能力，在多个数据集上达到当前最优或相当的性能，验证了预训练与后训练协同优化策略对于提升ViT模型FER性能的有效性。（本文相关代码可见https：//github.com/zfkarl/MIMIC/tree/master/Code_for_MIMIC）

Although unsupervised person re-identification (Re-ID) has drawn increasing research attention, it still faces the challenge of learning discriminative features in the absence of pairwise labels across disjoint camera views. To tackle the issue of label scarcity, researchers have delved into clustering and multilabel learning using memory dictionaries. Although effective in improving unsupervised Re-ID performance, these methods require substantial computational resources and introduce additional training complexity. To address this issue, we propose a conceptually simple yet effective and learnable module effective block, named the meta feature transformer (MFT). MFT is a streamlined, lightweight network architecture that operates without the need for complex networks or feature memory bank storage. It primarily focuses on learning interactions between sample features within small groups using a transformer mechanism in each mini-batch. It then generates a new sample feature for each group through a weighted sum. The main benefits of MFT arise from two aspects: (1) it allows for the use of numerous new samples for training, which significantly expands the feature space and enhances the network’s generalization capabilities; (2) the trainable attention weights highlight the importance of samples, enabling the network to focus on more useful or distinguishable samples. We validate our method on two popular large-scale Re-ID benchmarks, where extensive evaluations show that our MFT outperforms previous methods and significantly improves Re-ID performances.

With the deepening of research on Large Language Models (LLMs), significant progress has been made in recent years on the development of Large Multimodal Models (LMMs), which are gradually moving toward Artificial General Intelligence. This paper aims to summarize the recent progress from LLMs to LMMs in a comprehensive and unified way. First, we start with LLMs and outline various conceptual frameworks and key techniques. Then, we focus on the architectural components, training strategies, fine-tuning guidance, and prompt engineering of LMMs, and present a taxonomy of the latest vision–language LMMs. Finally, we provide a summary of both LLMs and LMMs from a unified perspective, make an analysis of the development status of large-scale models in the view of globalization, and offer potential research directions for large-scale models.

3D landmark detection plays a pivotal role in various applications such as 3D registration, pose estimation, and virtual try-on. While considerable success has been achieved in 2D human landmark detection or pose estimation, there is a notable scarcity of reported works on landmark detection in unordered 3D point clouds. This paper introduces a novel challenge, namely 3D landmark detection on human point clouds, presenting two primary contributions. Firstly, we establish a comprehensive human point cloud dataset, named HPoint103, designed to support the 3D landmark detection community. This dataset comprises 103 human point clouds created with commercial software and actors, each manually annotated with 11 stable landmarks. Secondly, we propose a Dual Cascade Point Transformer (D-CPT) model for precise point-based landmark detection. D-CPT gradually refines the landmarks through cascade Transformer decoder layers across the entire point cloud stream, simultaneously enhancing landmark coordinates with a RefineNet over local regions. Comparative evaluations with popular point-based methods on HPoint103 and the public dataset DHP19 demonstrate the dramatic outperformance of our D-CPT. Additionally, the integration of our RefineNet into existing methods consistently improves performance.

Multi-label image recognition is a task that predicts a set of object labels in an image. As the objects co-occur in the physical world, it is desirable to model label dependencies. Previous existing methods resort to either recurrent networks or pre-defined label correlation graphs for this purpose. In this paper, instead of using a pre-defined graph which is inflexible and may be sub-optimal for multi-label classification, we propose the A-GCN, which leverages the popular Graph Convolutional Networks with an Adaptive label correlation graph to model label dependencies. Specifically, we introduce a plug-and-play Label Graph (LG) module to learn label correlations with word embeddings, and then utilize traditional GCN to map this graph into label-dependent object classifiers which are further applied to image features. The basic LG module incorporates two 1x1 convolutional layers and uses the dot product to generate label graphs. In addition, we propose a sparse correlation constraint to enhance the LG module and also explore different LG architectures. We validate our method on two diverse multi-label datasets: MS-COCO and Fashion550K. Experimental results show that our A-GCN significantly improves baseline methods and achieves performance superior or comparable to the state of the art.

Despite the effectiveness of convolutional neural networks (CNNs) especially in image classification tasks, the effect of convolution features on learned representations is still limited. It mostly focuses on the salient object of the images, but ignores the variation information on clutter and local. In this paper, we propose a special framework, which is the multiple vector of locally aggregated descriptors (VLAD) encoding method with the CNNs features for image classification. Furthermore, in order to improve the performance of the VLAD coding method, we explore the multiplicity of VLAD encoding with the extension of three kinds of encoding algorithms, which are the VLAD-SA method, the VLAD-LSA and the VLAD-LLC method. Finally, we equip the spatial pyramid patch (SPM) on VLAD encoding to add the spatial information of CNNs feature. In particular, the power of SPM leads our framework to yield better performance compared to the existing method.