XZXinyu Zhu

Xinyu Zhu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Xinyu Zhu has a total of 11 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    BMV: Breaking Monolithic Verification for Fast Speculative LLM Inference

    by Dongxu Liu, Bing Zhu, Xinyu Zhu, Long Zheng, Xiaofei Liao, Hai Jin, Jingling Xue

  • Open Access

    HBSpec: A Hybrid-Bonding-based Heterogeneous Accelerator for Efficient Tree-Structured Speculative Decoding

    by Runze Wang, Qinggang Wang, Haifeng Liu, Xinyu Zhu, Chenggang Duan, Long Zheng, Xiaofei Liao, Hai Jin, Jingling Xue

    Emerging tree-structured speculative decoding has been proposed as a promising alternative to conventional sequence-structured speculative decoding, offering a higher verification success rate and reduced inference latency while exhibiting characteristics of higher computational intensity. However, existing near-memory processing (NMP)-enabled heterogeneous accelerators for large language models (LLMs) typically place processing elements (PEs) on DRAM dies, resulting in limited computation capability, which prevents them from fully accelerating tree-structured speculative decoding. In response, we propose a hybrid-bonding (HB)-based heterogeneous accelerator for tree-structured speculative decoding, called HBSpec, which customizes PEs on the incorporated logic die to enhance NMP computation capacity. HBSpec features a communication-optimized data mapping policy to minimize cross-bank data access overhead inherent in the distributed-memory design adopted in HB. Furthermore, an arithmetic intensity-aware operator scheduler is designed to dynamically assign operators to the most suitable hardware units, while a Tree-KV pool management scheme is introduced to eliminate memory fragmentation caused by discarded speculative branches. Experimental results demonstrate that HBSpec achieves average throughput improvements of 15.56×, 2.36×, and 2.13× over the state-of-the-art NPU-only, NPU+LPDDR, and NPU+HB baselines, respectively.