Xiaoxiao Jin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Xiaoxiao Jin has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Tuo Leng
School of Computer Engineering and Science Shanghai University Shanghai China
2 shared publications - Chenyang Mao
Department of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
1 shared publication - Dengfeng Yue
Department of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
1 shared publication Jia Zou
Shanghai University,School of Computer Engineering and Science,Shanghai,China
1 shared publication- Liangbo Xie
Chongqing University of Posts and Telecommunications
1 shared publication - Mu Zhou
Chongqing University of Posts and Telecommunications
1 shared publication
Latest Publications
- Open Access
Floating-Point Embedding: Enhancing the Mathematical Comprehension of Large Language Models
The processing and comprehension of numerical information in natural language represent pivotal focal points of scholarly inquiry. Across diverse applications spanning text analysis to information retrieval, the adept management and understanding of the numerical content within natural language are indispensable in achieving task success. Specialized encoding and embedding techniques tailored to numerical data offer an avenue toward improved performance in tasks involving masked prediction and numerical reasoning, inherently characterized by numerical values. Consequently, treating numbers in text merely as words is inadequate; their numerical semantics must be underscored. Recent years have witnessed the emergence of a range of specific encoding methodologies designed explicitly for numerical content, demonstrating promising outcomes. We observe similarities between the Transformer architecture and CPU architecture, with symmetry playing a crucial role. In light of this observation and drawing inspiration from computer system theory, we introduce a floating-point representation and devise a corresponding embedding module. The numerical representations correspond one-to-one with their semantic vector values, rendering both symmetric regarding intermediate transformation methods. Our proposed methodology facilitates the more comprehensive encoding and embedding of numerical information within a predefined precision range, thereby ensuring a distinctive encoding representation for each numerical entity. Rigorous testing on multiple encoder-only models and datasets yielded results that stand out in terms of competitiveness. In comparison to the default embedding methods employed by models, our approach achieved an improvement of approximately in Top-1 accuracy and a reduction in perplexity of approximately . These outcomes affirm the efficacy of our proposed method. Furthermore, the enrichment of numerical semantics through a more comprehensive embedding contributes to the augmentation of the model’s capacity for semantic understanding.Open AccessFloating-Point Embedding: Enhancing the Mathematical Comprehension of Large Language Models
The processing and comprehension of numerical information in natural language represent pivotal focal points of scholarly inquiry. Across diverse applications spanning text analysis to information retrieval, the adept management and understanding of the numerical content within natural language are indispensable in achieving task success. Specialized encoding and embedding techniques tailored to numerical data offer an avenue toward improved performance in tasks involving masked prediction and numerical reasoning, inherently characterized by numerical values. Consequently, treating numbers in text merely as words is inadequate; their numerical semantics must be underscored. Recent years have witnessed the emergence of a range of specific encoding methodologies designed explicitly for numerical content, demonstrating promising outcomes. We observe similarities between the Transformer architecture and CPU architecture, with symmetry playing a crucial role. In light of this observation and drawing inspiration from computer system theory, we introduce a floating-point representation and devise a corresponding embedding module. The numerical representations correspond one-to-one with their semantic vector values, rendering both symmetric regarding intermediate transformation methods. Our proposed methodology facilitates the more comprehensive encoding and embedding of numerical information within a predefined precision range, thereby ensuring a distinctive encoding representation for each numerical entity. Rigorous testing on multiple encoder-only models and datasets yielded results that stand out in terms of competitiveness. In comparison to the default embedding methods employed by models, our approach achieved an improvement of approximately in Top-1 accuracy and a reduction in perplexity of approximately . These outcomes affirm the efficacy of our proposed method. Furthermore, the enrichment of numerical semantics through a more comprehensive embedding contributes to the augmentation of the model’s capacity for semantic understanding. - 17 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 288-291
Efficient Model Compression in Super Resolution via Ghost Module with Minimal Angle Regularization
The past years have witnessed remarkable achievements in large-scale deep models. Nevertheless, their broad implementation on resource-constrained devices faces significant obstacles due to considerable computational complexity and extensive storage demands. To address these challenges, various approaches, such as model compression and neural architecture design, have been proposed. GhostNet, a lightweight network architecture, exploits the redundancy of feature maps by categorizing convolution filters into primary and cheap filters. Nonetheless, the limited number of primary filters, coupled with a high compression ratio, restrains the network’s expressiveness. Drawing inspiration from regularization methods targeting the angular diversity of neurons, we introduce a novel approach to augment GhostNet’s performance. Specifically, we propose the integration of maximizing minimal pairwise angles (MMA) regularization on primary filters. This augmentation aims to enhance the disparity between convolution kernels, thereby extracting more information at different levels. The MMA regularization strives to uniformly distribute the normalized weight vectors of neurons or filters on a hypersphere. Our experimental results, conducted on benchmark datasets, showcase the efficacy of the proposed method in enhancing GhostNet’s performance, particularly under higher compression ratios. This advancement opens new avenues for improving the deployment of deep models on resource-constrained devices.
- 17 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 288-291
Efficient Model Compression in Super Resolution via Ghost Module with Minimal Angle Regularization
The past years have witnessed remarkable achievements in large-scale deep models. Nevertheless, their broad implementation on resource-constrained devices faces significant obstacles due to considerable computational complexity and extensive storage demands. To address these challenges, various approaches, such as model compression and neural architecture design, have been proposed. GhostNet, a lightweight network architecture, exploits the redundancy of feature maps by categorizing convolution filters into primary and cheap filters. Nonetheless, the limited number of primary filters, coupled with a high compression ratio, restrains the network’s expressiveness. Drawing inspiration from regularization methods targeting the angular diversity of neurons, we introduce a novel approach to augment GhostNet’s performance. Specifically, we propose the integration of maximizing minimal pairwise angles (MMA) regularization on primary filters. This augmentation aims to enhance the disparity between convolution kernels, thereby extracting more information at different levels. The MMA regularization strives to uniformly distribute the normalized weight vectors of neurons or filters on a hypersphere. Our experimental results, conducted on benchmark datasets, showcase the efficacy of the proposed method in enhancing GhostNet’s performance, particularly under higher compression ratios. This advancement opens new avenues for improving the deployment of deep models on resource-constrained devices.
- 14 August 2019
- Published by Springer Nature
- p. 1303-1311
Trajectory Reckoning Method Based on BDS Attitude Measuring and Point Positioning
- 14 August 2019
- Published by Springer Nature
- p. 1303-1311
Trajectory Reckoning Method Based on BDS Attitude Measuring and Point Positioning