Xinyu Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xinyu Zhu has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Hai Jin
Huazhong University of Science and Technology
2 shared publications - Jingling Xue
University of New South Wales
2 shared publications - Xiaofei Liao
Huazhong University of Science and Technology
2 shared publications - Bing Zhu
HuaZhong University of Science and Technology, Wuhan, Hubei, China
1 shared publication - Chenggang Duan
HuaZhong University of Science and Technology, Wuhan, Hubei, China
1 shared publication - Dongxu Liu
HuaZhong University of Science and Technology, Wuhan, Hubei, China
1 shared publication
Latest Publications
- Open Access
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1467-1483
BMV: Breaking Monolithic Verification for Fast Speculative LLM Inference
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1467-1483
Open AccessBMV: Breaking Monolithic Verification for Fast Speculative LLM Inference
- Open Access
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
HBSpec: A Hybrid-Bonding-based Heterogeneous Accelerator for Efficient Tree-Structured Speculative Decoding
Emerging tree-structured speculative decoding has been proposed as a promising alternative to conventional sequence-structured speculative decoding, offering a higher verification success rate and reduced inference latency while exhibiting characteristics of higher computational intensity. However, existing near-memory processing (NMP)-enabled heterogeneous accelerators for large language models (LLMs) typically place processing elements (PEs) on DRAM dies, resulting in limited computation capability, which prevents them from fully accelerating tree-structured speculative decoding. In response, we propose a hybrid-bonding (HB)-based heterogeneous accelerator for tree-structured speculative decoding, called HBSpec, which customizes PEs on the incorporated logic die to enhance NMP computation capacity. HBSpec features a communication-optimized data mapping policy to minimize cross-bank data access overhead inherent in the distributed-memory design adopted in HB. Furthermore, an arithmetic intensity-aware operator scheduler is designed to dynamically assign operators to the most suitable hardware units, while a Tree-KV pool management scheme is introduced to eliminate memory fragmentation caused by discarded speculative branches. Experimental results demonstrate that HBSpec achieves average throughput improvements of 15.56×, 2.36×, and 2.13× over the state-of-the-art NPU-only, NPU+LPDDR, and NPU+HB baselines, respectively.
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessHBSpec: A Hybrid-Bonding-based Heterogeneous Accelerator for Efficient Tree-Structured Speculative Decoding
Emerging tree-structured speculative decoding has been proposed as a promising alternative to conventional sequence-structured speculative decoding, offering a higher verification success rate and reduced inference latency while exhibiting characteristics of higher computational intensity. However, existing near-memory processing (NMP)-enabled heterogeneous accelerators for large language models (LLMs) typically place processing elements (PEs) on DRAM dies, resulting in limited computation capability, which prevents them from fully accelerating tree-structured speculative decoding. In response, we propose a hybrid-bonding (HB)-based heterogeneous accelerator for tree-structured speculative decoding, called HBSpec, which customizes PEs on the incorporated logic die to enhance NMP computation capacity. HBSpec features a communication-optimized data mapping policy to minimize cross-bank data access overhead inherent in the distributed-memory design adopted in HB. Furthermore, an arithmetic intensity-aware operator scheduler is designed to dynamically assign operators to the most suitable hardware units, while a Tree-KV pool management scheme is introduced to eliminate memory fragmentation caused by discarded speculative branches. Experimental results demonstrate that HBSpec achieves average throughput improvements of 15.56×, 2.36×, and 2.13× over the state-of-the-art NPU-only, NPU+LPDDR, and NPU+HB baselines, respectively.