Zixin Wan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zixin Wan has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Tao Shen
Kunming University of Science and Technology
2 shared publications - HongWei Gu
School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
1 shared publication - Kai Zeng
Kunming University of Science and Technology
1 shared publication - Kaili Wang
Faculty of Information Engineering and Automation Kunming University of Science and Technology Kunming China
1 shared publication - Mingtao Wang
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
1 shared publication
Latest Publications
- Open Access
Binary Transformer Based on the Alignment and Correction of Distribution
Transformer is a powerful model widely used in artificial intelligence applications. It contains complex structures and has extremely high computational requirements that are not suitable for embedded intelligent sensors with limited computational resources. The binary quantization technology takes up less memory space and has a faster calculation speed; however, it is seldom studied for the lightweight transformer. Compared with full-precision networks, the key bottleneck lies in the distribution shift problem caused by the existing binary quantization methods. To tackle this problem, the feature distribution alignment operation in binarization is investigated. The median shift and mean restore is designed to ensure consistency between the binary feature distribution and the full-precision transformer. Then, a knowledge distillation architecture for distribution correction is developed, which has a teacher–student structure comprising a full-precision and binary transformer, to further rectify the feature distribution of the binary student network to ensure the completeness and accuracy of the data. Experimental results on the CIFAR10, CIFAR100, ImageNet-1k, and TinyImageNet datasets show the effectiveness of the proposed binary optimization model, which outperforms the previous state-of-the-art binarization mechanisms while maintaining the same computational complexity.
Open AccessBinary Transformer Based on the Alignment and Correction of Distribution
Transformer is a powerful model widely used in artificial intelligence applications. It contains complex structures and has extremely high computational requirements that are not suitable for embedded intelligent sensors with limited computational resources. The binary quantization technology takes up less memory space and has a faster calculation speed; however, it is seldom studied for the lightweight transformer. Compared with full-precision networks, the key bottleneck lies in the distribution shift problem caused by the existing binary quantization methods. To tackle this problem, the feature distribution alignment operation in binarization is investigated. The median shift and mean restore is designed to ensure consistency between the binary feature distribution and the full-precision transformer. Then, a knowledge distillation architecture for distribution correction is developed, which has a teacher–student structure comprising a full-precision and binary transformer, to further rectify the feature distribution of the binary student network to ensure the completeness and accuracy of the data. Experimental results on the CIFAR10, CIFAR100, ImageNet-1k, and TinyImageNet datasets show the effectiveness of the proposed binary optimization model, which outperforms the previous state-of-the-art binarization mechanisms while maintaining the same computational complexity.
- 1 March 2024
- Published by Springer Nature in Applied Intelligence
- Vol. 54 (6) , 4994-5014
Self-knowledge distillation enhanced binary neural networks derived from underutilized information
- 1 March 2024
- Published by Springer Nature in Applied Intelligence
- Vol. 54 (6) , 4994-5014
Self-knowledge distillation enhanced binary neural networks derived from underutilized information