Xinyu Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Xinyu Yang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Kai Chen
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Kaiqiang Xu
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Anoop Kumar
Amazon Alexa AI, Seattle, USA
1 shared publication - Han Tian
University of Science and Technology of China, Hefei, China
1 shared publication - Hoang Anh Just
Virginia Tech, Blacksburg, VA, USA
1 shared publication - Myeongseob Ko
Virginia Tech, United States of America
1 shared publication
Latest Publications
- Open Access
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 60-73
TurboBus: Pooling PCIe Bandwidth for LLM Workloads via Scale-Up Fabrics
GPU memory offloading is widely adopted for LLM workloads but shifts the bottleneck to GPU-CPU transfers, which can take up to 90% of the end-to-end inference/training time! Paradoxically, over 60% of PCIe bandwidth remains idle. The root cause is that PCIe links are individually bottlenecked but collectively underutilized. Bursty, phase-driven transfer patterns leave bandwidth idle both within and across jobs. We present TurboBus, which pools PCIe bandwidth across co-located jobs via emerging scale-up fabrics. TurboBus enables any GPU to borrow idle PCIe links from neighboring GPUs, even those belonging to other jobs, while preserving isolation through a privileged daemon. At the core of TurboBus, it streams data through relay GPUs with bounded memory overhead, keeps all links busy through fine-grained PCIe allocation, enables bidirectional transfers leveraging PCIe/NVLink bandwidth asymmetry, and balances fairness and completion time with a size-aware scheduling. We fully implement TurboBus and our experiments show that it reduces first-token latency by up to 40% for on-demand model loading (within 5% of the analytical optimum), achieves up to 1.6x throughput for KV-cache-offloaded inference, and accelerates training by up to 7%, while imposing less than 1% overhead on co-located workloads.
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 60-73
Open AccessTurboBus: Pooling PCIe Bandwidth for LLM Workloads via Scale-Up Fabrics
GPU memory offloading is widely adopted for LLM workloads but shifts the bottleneck to GPU-CPU transfers, which can take up to 90% of the end-to-end inference/training time! Paradoxically, over 60% of PCIe bandwidth remains idle. The root cause is that PCIe links are individually bottlenecked but collectively underutilized. Bursty, phase-driven transfer patterns leave bandwidth idle both within and across jobs. We present TurboBus, which pools PCIe bandwidth across co-located jobs via emerging scale-up fabrics. TurboBus enables any GPU to borrow idle PCIe links from neighboring GPUs, even those belonging to other jobs, while preserving isolation through a privileged daemon. At the core of TurboBus, it streams data through relay GPUs with bounded memory overhead, keeps all links busy through fine-grained PCIe allocation, enables bidirectional transfers leveraging PCIe/NVLink bandwidth asymmetry, and balances fairness and completion time with a size-aware scheduling. We fully implement TurboBus and our experiments show that it reduces first-token latency by up to 40% for on-demand model loading (within 5% of the analytical optimum), achieves up to 1.6x throughput for KV-cache-offloaded inference, and accelerates training by up to 7%, while imposing less than 1% overhead on co-located workloads.
- Open Access
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 1232-1234
Coflow Scheduling for LLM Training
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 1232-1234
Open AccessCoflow Scheduling for LLM Training
- Open Access
- 16 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 264-281
PrivMon: A Stream-Based System for Real-Time Privacy Attack Detection for Machine Learning Models
- 16 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 264-281
Open AccessPrivMon: A Stream-Based System for Real-Time Privacy Attack Detection for Machine Learning Models