Xudong Ma
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Xudong Ma has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Jie Luo
Beihang University
5 shared publications - Haotong Qin
ETH Zurich
3 shared publications - Xianglong Liu
Beihang University, Beijing, China
3 shared publications - Michele Magno
ETH Zurich
2 shared publications - Tengwei Song
Beihang University
2 shared publications - Xingyu Zheng
School of Computer Science and Engineering, Beihang University, Beijing, China
2 shared publications
Latest Publications
- 7 May 2026
- Published by Springer Nature
- p. 417-435
Robust Knowledge Graph Embedding via Denoising
- Open Access
- 16 April 2026
- Published by Springer Nature in Visual Intelligence
- Vol. 4 (1)
An empirical study of Qwen3 quantization
The Qwen series has emerged as a leading family of open-source large language models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the recent release of Qwen3, which exhibits superior performance across diverse benchmarks, there is an increased interest in the efficient deployment of these models in resource-constrained environments. Low-bit quantization presents a promising solution, yet its impact on Qwen3’s performance remains underexplored. This study conducts a systematic evaluation of Qwen3’s robustness under various quantization settings, aiming to identify both the opportunities and the challenges inherent in compressing this state-of-the-art model. We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their effectiveness across multiple datasets. Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision, underscoring the persistent hurdles in LLM compression. These results emphasize the need for further research to mitigate performance loss in extreme quantization scenarios. We anticipate that this empirical analysis will provide actionable insights for advancing quantization methods tailored to Qwen3 and future LLMs, ultimately enhancing their practicality without compromising accuracy.
- 16 April 2026
- Published by Springer Nature in Visual Intelligence
- Vol. 4 (1)
Open AccessAn empirical study of Qwen3 quantization
The Qwen series has emerged as a leading family of open-source large language models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the recent release of Qwen3, which exhibits superior performance across diverse benchmarks, there is an increased interest in the efficient deployment of these models in resource-constrained environments. Low-bit quantization presents a promising solution, yet its impact on Qwen3’s performance remains underexplored. This study conducts a systematic evaluation of Qwen3’s robustness under various quantization settings, aiming to identify both the opportunities and the challenges inherent in compressing this state-of-the-art model. We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their effectiveness across multiple datasets. Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision, underscoring the persistent hurdles in LLM compression. These results emphasize the need for further research to mitigate performance loss in extreme quantization scenarios. We anticipate that this empirical analysis will provide actionable insights for advancing quantization methods tailored to Qwen3 and future LLMs, ultimately enhancing their practicality without compromising accuracy.
- Open Access
- 11 September 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 43 (6) , 1-38
Deep Hashing with Semantic Hash Centers for Image Retrieval
Deep hashing presents an effective strategy for large-scale image retrieval. Current hashing methods are generally categorized by their supervision types: point-wise, pairwise, and list-wise. Recent advancements in point-wise methods (e.g., CSQ, MDS) have significantly enhanced retrieval performance across diverse datasets by pre-assigning a hash center to each class, thereby improving the discriminability of the resultant hash codes. However, these methods employ purely data-independent algorithms for generating hash centers, overlooking the semantic connections between different classes, which, we argue, could degrade retrieval performance. To tackle this problem, this article expands on the newly emerged concept of “hash centers” to introduce “ semantic hash centers,” which posits that hash centers of semantically related classes should exhibit closer Hamming distances, while those of unrelated classes should be more distant. Based on this hypothesis, we propose a three-stage framework, termed Semantic Hash Centers (SHC), to produce hash codes that preserve semantics. First, we build a classification network to detect semantic similarities between classes, and utilize a data-dependent approach to similarity calculation that can adapt to varied data distributions. Next, we develop a new optimization algorithm to generate SHC. This algorithm not only maintains semantic relatedness among hash centers but also integrates a constraint to ensure a minimum distance between them, addressing the issue of excessively proximate hash centers potentially impairing retrieval performance. Finally, we train a deep hashing network with the above generated SHC to convert each image into a binary hash code. Experiments on large-scale image retrieval across several public datasets demonstrate that SHC generates more discriminative hash codes, markedly enhancing retrieval performance. Specifically, in terms of the mAP@100, mAP@1000, and mAP@ALL metrics, SHC records average improvements of +6.24%, +6.68%, and +10.39%, respectively, over the most competitive existing methods. The code of our SHC project is available at https://github.com/cc752424640/Deep-Hashing-with-Semantic-Hash-Centers-for-Image-Retrieval .
- 11 September 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 43 (6) , 1-38
Open AccessDeep Hashing with Semantic Hash Centers for Image Retrieval
Deep hashing presents an effective strategy for large-scale image retrieval. Current hashing methods are generally categorized by their supervision types: point-wise, pairwise, and list-wise. Recent advancements in point-wise methods (e.g., CSQ, MDS) have significantly enhanced retrieval performance across diverse datasets by pre-assigning a hash center to each class, thereby improving the discriminability of the resultant hash codes. However, these methods employ purely data-independent algorithms for generating hash centers, overlooking the semantic connections between different classes, which, we argue, could degrade retrieval performance. To tackle this problem, this article expands on the newly emerged concept of “hash centers” to introduce “ semantic hash centers,” which posits that hash centers of semantically related classes should exhibit closer Hamming distances, while those of unrelated classes should be more distant. Based on this hypothesis, we propose a three-stage framework, termed Semantic Hash Centers (SHC), to produce hash codes that preserve semantics. First, we build a classification network to detect semantic similarities between classes, and utilize a data-dependent approach to similarity calculation that can adapt to varied data distributions. Next, we develop a new optimization algorithm to generate SHC. This algorithm not only maintains semantic relatedness among hash centers but also integrates a constraint to ensure a minimum distance between them, addressing the issue of excessively proximate hash centers potentially impairing retrieval performance. Finally, we train a deep hashing network with the above generated SHC to convert each image into a binary hash code. Experiments on large-scale image retrieval across several public datasets demonstrate that SHC generates more discriminative hash codes, markedly enhancing retrieval performance. Specifically, in terms of the mAP@100, mAP@1000, and mAP@ALL metrics, SHC records average improvements of +6.24%, +6.68%, and +10.39%, respectively, over the most competitive existing methods. The code of our SHC project is available at https://github.com/cc752424640/Deep-Hashing-with-Semantic-Hash-Centers-for-Image-Retrieval .
- Top Cited PapersOpen Access
- 30 December 2024
- Published by Springer Nature in Visual Intelligence
- Vol. 2 (1) , 1-13
An empirical study of LLaMA3 quantization: from LLMs to MLLMs
The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the popular LLM backbone of multi-modal large language models (MLLMs), widely used in computer vision and natural language understanding tasks. In particular, LLaMA3 models have recently been released and have achieved impressive performance in various domains with super-large scale pre-training on over 15T tokens of data. Given the wide application of low-bit quantization for LLMs in resource-constrained scenarios, we explore LLaMA3’s capabilities when quantized to low bit-width. This exploration can potentially provide new insights and challenges for the low-bit quantization of LLaMA3 and other future LLMs, especially in addressing performance degradation issues that suffer in LLM compression. Specifically, we comprehensively evaluate the 10 existing post-training quantization and LoRA fine-tuning (LoRA-FT) methods of LLaMA3 on 1-8 bits and various datasets to reveal the low-bit quantization performance of LLaMA3. To uncover the capabilities of low-bit quantized MLLM, we assessed the performance of the LLaMA3-based LLaVA-Next-8B model under 2-4 ultra-low bits with post-training quantization methods. Our experimental results indicate that LLaMA3 still suffers from non-negligible degradation in linguistic and visual contexts, particularly under ultra-low bit widths. This highlights the significant performance gap at low bit-width that needs to be addressed in future developments. We expect that this empirical study will prove valuable in advancing future models, driving LLMs and MLLMs to achieve higher accuracy at lower bit to enhance practicality.
- 30 December 2024
- Published by Springer Nature in Visual Intelligence
- Vol. 2 (1) , 1-13
Top Cited PapersOpen AccessAn empirical study of LLaMA3 quantization: from LLMs to MLLMs
The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the popular LLM backbone of multi-modal large language models (MLLMs), widely used in computer vision and natural language understanding tasks. In particular, LLaMA3 models have recently been released and have achieved impressive performance in various domains with super-large scale pre-training on over 15T tokens of data. Given the wide application of low-bit quantization for LLMs in resource-constrained scenarios, we explore LLaMA3’s capabilities when quantized to low bit-width. This exploration can potentially provide new insights and challenges for the low-bit quantization of LLaMA3 and other future LLMs, especially in addressing performance degradation issues that suffer in LLM compression. Specifically, we comprehensively evaluate the 10 existing post-training quantization and LoRA fine-tuning (LoRA-FT) methods of LLaMA3 on 1-8 bits and various datasets to reveal the low-bit quantization performance of LLaMA3. To uncover the capabilities of low-bit quantized MLLM, we assessed the performance of the LLaMA3-based LLaVA-Next-8B model under 2-4 ultra-low bits with post-training quantization methods. Our experimental results indicate that LLaMA3 still suffers from non-negligible degradation in linguistic and visual contexts, particularly under ultra-low bit widths. This highlights the significant performance gap at low bit-width that needs to be addressed in future developments. We expect that this empirical study will prove valuable in advancing future models, driving LLMs and MLLMs to achieve higher accuracy at lower bit to enhance practicality.
- 9 August 2023
- Published by Springer Nature
- p. 379-392
Dual Channel Knowledge Graph Embedding with Ontology Guided Data Augmentation
- 9 August 2023
- Published by Springer Nature
- p. 379-392
Dual Channel Knowledge Graph Embedding with Ontology Guided Data Augmentation
- 22 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10674-10686
BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance
Deep neural networks, such as the deep-FSMN, have been widely studied for keyword spotting (KWS) applications while suffering expensive computation and storage. Therefore, network compression technologies such as binarization are studied to deploy KWS models on edge. In this article, we present a strong yet efficient binary neural network for KWS, namely, BiFSMNv2, pushing it to the real-network accuracy performance. First, we present a dual-scale thinnable 1-bit-architecture (DTA) to recover the representation capability of the binarized computation units by dual-scale activation binarization and liberate the speedup potential from an overall architecture perspective. Second, we also construct a frequency-independent distillation (FID) scheme for KWS binarization-aware training, which distills the high-and low-frequency components independently to mitigate the information mismatch between full-precision and binarized representations. Moreover, we propose the learning propagation binarizer (LPB), a general and efficient binarizer that enables the forward and backward propagation of binary KWS networks to be continuously improved through learning. We implement and deploy BiFSMNv2 on ARMv8 real-world hardware with a novel fast bitwise computation kernel (FBCK), which is proposed to fully use registers and increase instruction throughput. Comprehensive experiments show our BiFSMNv2 outperforms the existing binary networks for KWS by convincing margins across different datasets and achieves comparable accuracy with the full-precision networks (only a tiny 1.51% drop on Speech Commands V1–12). We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25.1 $\times$ speedup and 20.2 $\times$ storage-saving on edge hardware.
- 22 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10674-10686
BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance
Deep neural networks, such as the deep-FSMN, have been widely studied for keyword spotting (KWS) applications while suffering expensive computation and storage. Therefore, network compression technologies such as binarization are studied to deploy KWS models on edge. In this article, we present a strong yet efficient binary neural network for KWS, namely, BiFSMNv2, pushing it to the real-network accuracy performance. First, we present a dual-scale thinnable 1-bit-architecture (DTA) to recover the representation capability of the binarized computation units by dual-scale activation binarization and liberate the speedup potential from an overall architecture perspective. Second, we also construct a frequency-independent distillation (FID) scheme for KWS binarization-aware training, which distills the high-and low-frequency components independently to mitigate the information mismatch between full-precision and binarized representations. Moreover, we propose the learning propagation binarizer (LPB), a general and efficient binarizer that enables the forward and backward propagation of binary KWS networks to be continuously improved through learning. We implement and deploy BiFSMNv2 on ARMv8 real-world hardware with a novel fast bitwise computation kernel (FBCK), which is proposed to fully use registers and increase instruction throughput. Comprehensive experiments show our BiFSMNv2 outperforms the existing binary networks for KWS by convincing margins across different datasets and achieves comparable accuracy with the full-precision networks (only a tiny 1.51% drop on Speech Commands V1–12). We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25.1 $\times$ speedup and 20.2 $\times$ storage-saving on edge hardware.