Jiaan Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jiaan Zhu has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Cheng Li
University of Science and Technology of China, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
4 shared publications - Youhui Bai
University of Science and Technology of China, Hefei, China
4 shared publications - Long Zhao
Anhui University, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
3 shared publications - Qinghe Wang
University of Science and Technology of China, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
3 shared publications - Zewen Jin
University of Science and Technology of China, Hefei, China
3 shared publications - Chaoyi Ruan
National University of Singapore, Singapore, Singapore
2 shared publications
Latest Publications
- Open Access
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1941-1958
DittoCache: From CPU-Orchestrated to GPU-Centric KVCache for Efficient LLM Decoding
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1941-1958
Open AccessDittoCache: From CPU-Orchestrated to GPU-Centric KVCache for Efficient LLM Decoding
- Open Access
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-15
ATP: Workload-Adaptive Scheduling for Goodput-Optimized Tensor-Parallel LLM Serving with PD Disaggregation
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Open AccessATP: Workload-Adaptive Scheduling for Goodput-Optimized Tensor-Parallel LLM Serving with PD Disaggregation
- Open Access
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 1092-1102
Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality. However, the synchronous three-stage RLHF pipeline is often bottlenecked by the generation stage, where response-length skew causes the effective batch size to shrink rapidly during decoding, leaving GPUs underutilized while a few long responses remain unfinished. Mainstream frameworks employ a static tensor parallelism (TP) configuration that cannot adapt to changing batch characteristics, leaving substantial performance headroom unexplored. We propose PAT, an adaptive TP method that dynamically reconfigures TP during the generation stage of each RLHF iteration. PAT introduces two key techniques. First, a predictor-guided online reconfiguration method decides both the reconfiguration point and the target TP configuration based on offline profiling, triggering reconfiguration only when the predicted latency benefit outweighs the reconfiguration overhead. Second, a lightweight online reconfiguration mechanism updates only the states and layouts affected by TP changes: it adapts unfinished decoding states through a cost-model-based choice between KV-cache migration and recomputation, performs in-place weight resharding, and reuses cached communication groups. We implement PAT on top of SGLang and integrate it with the VeRL framework. Evaluations on LLaMA3.1-8B and Qwen3-14B using DeepScaleR show that PAT reduces generation latency by up to 34.6% and end-to-end RLHF training iteration latency by up to 27.2% compared to the original VeRL setup.
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 1092-1102
Open AccessAccelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality. However, the synchronous three-stage RLHF pipeline is often bottlenecked by the generation stage, where response-length skew causes the effective batch size to shrink rapidly during decoding, leaving GPUs underutilized while a few long responses remain unfinished. Mainstream frameworks employ a static tensor parallelism (TP) configuration that cannot adapt to changing batch characteristics, leaving substantial performance headroom unexplored. We propose PAT, an adaptive TP method that dynamically reconfigures TP during the generation stage of each RLHF iteration. PAT introduces two key techniques. First, a predictor-guided online reconfiguration method decides both the reconfiguration point and the target TP configuration based on offline profiling, triggering reconfiguration only when the predicted latency benefit outweighs the reconfiguration overhead. Second, a lightweight online reconfiguration mechanism updates only the states and layouts affected by TP changes: it adapts unfinished decoding states through a cost-model-based choice between KV-cache migration and recomputation, performs in-place weight resharding, and reuses cached communication groups. We implement PAT on top of SGLang and integrate it with the VeRL framework. Evaluations on LLaMA3.1-8B and Qwen3-14B using DeepScaleR show that PAT reduces generation latency by up to 34.6% and end-to-end RLHF training iteration latency by up to 27.2% compared to the original VeRL setup.