Mo Zhou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mo Zhou has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Chao Fang
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
2 shared publications - Chen Tian
Nanjing University, Nanjing, China
2 shared publications - Chengying Huan
Nanjing University, Nanjing, China
2 shared publications - Guihai Chen
Shanghai Jiao Tong University
2 shared publications - Kun Yang
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China
2 shared publications - Rong Gu
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China
2 shared publications
Latest Publications
- Open Access
- 24 September 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (4) , 1-2
Corrigendum for CoDec: Prefix-Shared Decoding Kernel for LLMs
This is a corrigendum for the article ''CoDec: Prefix-Shared Decoding Kernel for LLMs'' published in Proc. ACM Manag. Data 4, 3 (SIGMOD), Article 151 (June 2026), 27 pages.
- 24 September 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (4) , 1-2
Open AccessCorrigendum for CoDec: Prefix-Shared Decoding Kernel for LLMs
This is a corrigendum for the article ''CoDec: Prefix-Shared Decoding Kernel for LLMs'' published in Proc. ACM Manag. Data 4, 3 (SIGMOD), Article 151 (June 2026), 27 pages.
- Open Access
- 18 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (3) , 1-27
CoDec: Prefix-Shared Decoding Kernel for LLMs
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critical bottleneck with increasing context lengths, is a memory-intensive process requiring heavy memory access on the key-value (KV) cache of the prefixes. Therefore, in this paper, we explore the potential of prefix-sharing in the attention computation of the decode stage. However, the tree structure of the prefix-sharing mechanism presents significant challenges for attention computation in efficiently processing shared KV cache access patterns while managing complex dependencies and balancing irregular workloads. To address the above challenges, we propose a dedicated attention kernel to co mbine the memory access of shared prefixes in the dec oding stage, namely CoDec. CoDec delivers two key innovations: a novel shared-prefix attention kernel that optimizes memory hierarchy and exploits both intra-block and inter-block parallelism, and a comprehensive workload balancing mechanism that efficiently estimates cost, divides tasks, and schedules execution. Experimental results show that CoDec achieves an average 1.9× speedup and 120.9× memory access reduction compared to the state-of-the-art FlashDecoding kernel regarding attention computation in the decode stage and 3.8× end-to-end time per output token compared to vLLM.
- 18 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (3) , 1-27
Open AccessCoDec: Prefix-Shared Decoding Kernel for LLMs
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critical bottleneck with increasing context lengths, is a memory-intensive process requiring heavy memory access on the key-value (KV) cache of the prefixes. Therefore, in this paper, we explore the potential of prefix-sharing in the attention computation of the decode stage. However, the tree structure of the prefix-sharing mechanism presents significant challenges for attention computation in efficiently processing shared KV cache access patterns while managing complex dependencies and balancing irregular workloads. To address the above challenges, we propose a dedicated attention kernel to co mbine the memory access of shared prefixes in the dec oding stage, namely CoDec. CoDec delivers two key innovations: a novel shared-prefix attention kernel that optimizes memory hierarchy and exploits both intra-block and inter-block parallelism, and a comprehensive workload balancing mechanism that efficiently estimates cost, divides tasks, and schedules execution. Experimental results show that CoDec achieves an average 1.9× speedup and 120.9× memory access reduction compared to the state-of-the-art FlashDecoding kernel regarding attention computation in the decode stage and 3.8× end-to-end time per output token compared to vLLM.