Kaili Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kaili Wang has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
Fenhua Bai
Faculty of Information Engineering and Automation Kunming University of Science and Technology Kunming China
1 shared publicationHuijie He
Faculty of Information Engineering and Automation Kunming University of Science and Technology Kunming China
1 shared publication- Mingtao Wang
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
1 shared publication Tao Shen
Faculty of Information Engineering and Automation Kunming University of Science and Technology Kunming China
1 shared publication- Tao Shen
Kunming University of Science and Technology
1 shared publication Xinwei Sun
School of Intelligence Science and Technology Tongji University Shanghai China
1 shared publication
Latest Publications
- Open Access
- 15 July 2025
- Published by Institution of Engineering and Technology (IET) in CAAI Transactions on Intelligence Technology
- Vol. 10 (5) , 1459-1477
Approximate‐Guided Representation Learning in Vision Transformer
In recent years, the transformer model has demonstrated excellent performance in computer vision (CV) applications. The key lies in its guided representation attention mechanism, which uses dot‐product to depict complex feature relationships, and comprehensively understands the context semantics to obtain feature weights. Then feature enhancement is implemented by guiding the target matrix through feature weights. However, the uncertainty and inconsistency of features are widespread that prone to confusion in the description of relationships within dot‐product attention mechanisms. To solve this problem, this paper proposed a novel approximate‐guided representation learning methodology for vision transformer. The kernelised matroids fuzzy rough set is defined, wherein the closed sets inside kernelised fuzzy information granules of matroids structures can constitute the subspace of lower approximation in rough sets. Thus, the kernel relation is employed to characterise image feature granules that will be reconstructed according to the independent set in matroids theory. Then, according to the characteristics of the closed set within matroids, the feature attention weight is formed by using the lower approximation to realise the approximate guidance of features. The approximate‐guided representation mechanism can be flexibly deployed as a plug‐and‐play component in a wide range of CV tasks. Extensive empirical results demonstrate that the proposed method outperforms the majority of advanced prevalent models, especially in terms of robustness.
- 15 July 2025
- Published by Institution of Engineering and Technology (IET) in CAAI Transactions on Intelligence Technology
- Vol. 10 (5) , 1459-1477
Open AccessApproximate‐Guided Representation Learning in Vision Transformer
In recent years, the transformer model has demonstrated excellent performance in computer vision (CV) applications. The key lies in its guided representation attention mechanism, which uses dot‐product to depict complex feature relationships, and comprehensively understands the context semantics to obtain feature weights. Then feature enhancement is implemented by guiding the target matrix through feature weights. However, the uncertainty and inconsistency of features are widespread that prone to confusion in the description of relationships within dot‐product attention mechanisms. To solve this problem, this paper proposed a novel approximate‐guided representation learning methodology for vision transformer. The kernelised matroids fuzzy rough set is defined, wherein the closed sets inside kernelised fuzzy information granules of matroids structures can constitute the subspace of lower approximation in rough sets. Thus, the kernel relation is employed to characterise image feature granules that will be reconstructed according to the independent set in matroids theory. Then, according to the characteristics of the closed set within matroids, the feature attention weight is formed by using the lower approximation to realise the approximate guidance of features. The approximate‐guided representation mechanism can be flexibly deployed as a plug‐and‐play component in a wide range of CV tasks. Extensive empirical results demonstrate that the proposed method outperforms the majority of advanced prevalent models, especially in terms of robustness.
- Open Access
Binary Transformer Based on the Alignment and Correction of Distribution
Transformer is a powerful model widely used in artificial intelligence applications. It contains complex structures and has extremely high computational requirements that are not suitable for embedded intelligent sensors with limited computational resources. The binary quantization technology takes up less memory space and has a faster calculation speed; however, it is seldom studied for the lightweight transformer. Compared with full-precision networks, the key bottleneck lies in the distribution shift problem caused by the existing binary quantization methods. To tackle this problem, the feature distribution alignment operation in binarization is investigated. The median shift and mean restore is designed to ensure consistency between the binary feature distribution and the full-precision transformer. Then, a knowledge distillation architecture for distribution correction is developed, which has a teacher–student structure comprising a full-precision and binary transformer, to further rectify the feature distribution of the binary student network to ensure the completeness and accuracy of the data. Experimental results on the CIFAR10, CIFAR100, ImageNet-1k, and TinyImageNet datasets show the effectiveness of the proposed binary optimization model, which outperforms the previous state-of-the-art binarization mechanisms while maintaining the same computational complexity.
Open AccessBinary Transformer Based on the Alignment and Correction of Distribution
Transformer is a powerful model widely used in artificial intelligence applications. It contains complex structures and has extremely high computational requirements that are not suitable for embedded intelligent sensors with limited computational resources. The binary quantization technology takes up less memory space and has a faster calculation speed; however, it is seldom studied for the lightweight transformer. Compared with full-precision networks, the key bottleneck lies in the distribution shift problem caused by the existing binary quantization methods. To tackle this problem, the feature distribution alignment operation in binarization is investigated. The median shift and mean restore is designed to ensure consistency between the binary feature distribution and the full-precision transformer. Then, a knowledge distillation architecture for distribution correction is developed, which has a teacher–student structure comprising a full-precision and binary transformer, to further rectify the feature distribution of the binary student network to ensure the completeness and accuracy of the data. Experimental results on the CIFAR10, CIFAR100, ImageNet-1k, and TinyImageNet datasets show the effectiveness of the proposed binary optimization model, which outperforms the previous state-of-the-art binarization mechanisms while maintaining the same computational complexity.