Vincent Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Vincent Liu has a total of 59 co-authors. Here are the top co-authors by number of shared publications:
- Liangcheng Yu
Microsoft Research, Redmond, Washington, USA
3 shared publications - Behnaz Arzani
Microsoft Research
2 shared publications - Boon Thau Loo
University of Pennsylvania, Philadelphia, PA, USA
2 shared publications - Dan Li
Tsinghua University, Beijing, Beijing, China
2 shared publications - Kaihui Gao
Zhongguancun Laboratory, Beijing, China
2 shared publications - Kelvin K. W. Ng
University of Pennsylvania, Philadelphia, Pennsylvania, USA
2 shared publications
Latest Publications
- Open Access
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 1170-1184
Honey, I Shrunk the Headers With Flow.ZIP
Packet header overhead is a persistent source of inefficiency in packet-switched networks, reducing goodput and increasing network load. Trends like tunneling further increase this overhead, significantly impacting flow completion times. While, in principle, it is possible to compress these headers, existing methods require specialized hardware on every hop to compress/decompress the packet to/from custom header formats. In this paper, we present Flow.zip, a backward-compatible header compression mechanism designed for existing data center networks. Our solution leverages a combination of last-hop network offload and MPLS support, both of which are ubiquitous in modern data center deployments. Flow.zip overcomes scalability limitations in these components by selectively and intelligently coordinating compression for a subset of flows. Doing so, Flow.zip achieves up to 58% reduction in average flow completion time on real-world data center workloads.
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 1170-1184
Open AccessHoney, I Shrunk the Headers With Flow.ZIP
Packet header overhead is a persistent source of inefficiency in packet-switched networks, reducing goodput and increasing network load. Trends like tunneling further increase this overhead, significantly impacting flow completion times. While, in principle, it is possible to compress these headers, existing methods require specialized hardware on every hop to compress/decompress the packet to/from custom header formats. In this paper, we present Flow.zip, a backward-compatible header compression mechanism designed for existing data center networks. Our solution leverages a combination of last-hop network offload and MPLS support, both of which are ubiquitous in modern data center deployments. Flow.zip overcomes scalability limitations in these components by selectively and intelligently coordinating compression for a subset of flows. Doing so, Flow.zip achieves up to 58% reduction in average flow completion time on real-world data center workloads.
- Open Access
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 46-59
Connex: Endpoint Mobility Primitives for Dynamic LLM Serving
Modern LLM serving systems increasingly adopt elastic inference pipelines where stages frequently join, leave, and migrate across nodes. However, existing GPU communication frameworks like NCCL assume static topologies, causing routing failures and P99 latency spikes during worker transitions that violate sub-millisecond tail latency requirements. We present Connex, a communication system that elevates endpoint mobility from exceptional failure to first-class primitive. Rather than optimizing individual mechanisms in isolation, Connex defines a mobility contract that the communication layer enforces whenever workers join, leave, or migrate while token streams, activations, or KV transfers are in flight. The contract is realized through three cooperating mechanisms: (1) epoch-based routing that bounds staleness without global coordination, (2) explicit handover protocols that preserve stream ordering and provide exactly-once delivery across migrations, and (3) credit-based backpressure with traffic-class isolation that prevents churn-induced interference with latency-critical paths. Evaluation on a 5-node GPU cluster under synthetic and production-derived churn shows that Connex reduces P99 tail spikes by up to 85% compared to NCCL-based baselines, achieves sub-second cutover, and maintains 100% goodput at moderate loads where baselines collapse to 0–28%, while incurring less than 5% steady-state overhead.
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 46-59
Open AccessConnex: Endpoint Mobility Primitives for Dynamic LLM Serving
Modern LLM serving systems increasingly adopt elastic inference pipelines where stages frequently join, leave, and migrate across nodes. However, existing GPU communication frameworks like NCCL assume static topologies, causing routing failures and P99 latency spikes during worker transitions that violate sub-millisecond tail latency requirements. We present Connex, a communication system that elevates endpoint mobility from exceptional failure to first-class primitive. Rather than optimizing individual mechanisms in isolation, Connex defines a mobility contract that the communication layer enforces whenever workers join, leave, or migrate while token streams, activations, or KV transfers are in flight. The contract is realized through three cooperating mechanisms: (1) epoch-based routing that bounds staleness without global coordination, (2) explicit handover protocols that preserve stream ordering and provide exactly-once delivery across migrations, and (3) credit-based backpressure with traffic-class isolation that prevents churn-induced interference with latency-critical paths. Evaluation on a 5-node GPU cluster under synthetic and production-derived churn shows that Connex reduces P99 tail spikes by up to 85% compared to NCCL-based baselines, achieves sub-second cutover, and maintains 100% goodput at moderate loads where baselines collapse to 0–28%, while incurring less than 5% steady-state overhead.
- Open Access
- 19 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 735-747
Multiplexed Heterogeneous LLM Serving via Stage-Aligned Parallelism
- 19 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 735-747
Open AccessMultiplexed Heterogeneous LLM Serving via Stage-Aligned Parallelism
- Open Access
- 17 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 131-138
Lost in Translation: The Search for Meaning in Network-Attached AI Accelerator Disaggregation
- 17 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 131-138
Open AccessLost in Translation: The Search for Meaning in Network-Attached AI Accelerator Disaggregation
- Open Access
- 6 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 129-146
λ-trim: Optimizing Function Initialization in Serverless Applications With Cost-driven Debloating
- 6 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 129-146
Open Accessλ-trim: Optimizing Function Initialization in Serverless Applications With Cost-driven Debloating
- Top Cited PapersOpen Access
- 4 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 16-37
Rethinking Machine Learning Collective Communication as a Multi-Commodity Flow Problem
Cloud operators utilize collective communication optimizers to enhance the efficiency of the single-tenant, centrally managed training clusters they manage. However, current optimizers struggle to scale for such use cases and often compromise solution quality for scalability. Our solution, TE-CCL, adopts a traffic-engineering-based approach to collective communication. Compared to a state-of-the-art optimizer, TACCL, TE-CCL produced schedules with 2× better performance on topologies TACCL supports (and its solver took a similar amount of time as TACCL's heuristic-based approach). TECCL additionally scales to larger topologies than TACCL. On our GPU testbed, TE-CCL outperformed TACCL by 2.14× and RCCL by 3.18× in terms of algorithm bandwidth.
- 4 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 16-37
Top Cited PapersOpen AccessRethinking Machine Learning Collective Communication as a Multi-Commodity Flow Problem
Cloud operators utilize collective communication optimizers to enhance the efficiency of the single-tenant, centrally managed training clusters they manage. However, current optimizers struggle to scale for such use cases and often compromise solution quality for scalability. Our solution, TE-CCL, adopts a traffic-engineering-based approach to collective communication. Compared to a state-of-the-art optimizer, TACCL, TE-CCL produced schedules with 2× better performance on topologies TACCL supports (and its solver took a similar amount of time as TACCL's heuristic-based approach). TECCL additionally scales to larger topologies than TACCL. On our GPU testbed, TE-CCL outperformed TACCL by 2.14× and RCCL by 3.18× in terms of algorithm bandwidth.
- Open Access
- 23 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 595-610
Paella: Low-latency Model Serving with Software-defined GPU Scheduling
Model serving systems play a critical role in multiplexing machine learning inference jobs across shared GPU infrastructure. These systems have traditionally sat at a high level of abstraction---receiving jobs from clients through a narrow API and relying on black-box GPU scheduling mechanisms when dispatching them. Fundamental limitations in the built-in GPU hardware scheduler, in particular, can lead to inefficiency when executing concurrent jobs. The current abstraction level also incurs system overheads that are similarly most significant when the GPU is heavily shared. In this paper, we argue for co-designing the model compiler, local clients, and the scheduler to bypass the built-in GPU scheduler and enable software control of kernel execution order. Doing so enables the use of arbitrary scheduling algorithms and reduces system overheads throughout the critical path of inference.
- 23 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 595-610
Open AccessPaella: Low-latency Model Serving with Software-defined GPU Scheduling
Model serving systems play a critical role in multiplexing machine learning inference jobs across shared GPU infrastructure. These systems have traditionally sat at a high level of abstraction---receiving jobs from clients through a narrow API and relying on black-box GPU scheduling mechanisms when dispatching them. Fundamental limitations in the built-in GPU hardware scheduler, in particular, can lead to inefficiency when executing concurrent jobs. The current abstraction level also incurs system overheads that are similarly most significant when the GPU is heavily shared. In this paper, we argue for co-designing the model compiler, local clients, and the scheduler to bypass the built-in GPU scheduler and enable software control of kernel execution order. Doing so enables the use of arbitrary scheduling algorithms and reduces system overheads throughout the critical path of inference.
- Open Access
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 1179-1181
Demo: NetVision: Efficient Visualization Front-End for Packet-level Discrete-Event Network Simulation
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 1179-1181
Open AccessDemo: NetVision: Efficient Visualization Front-End for Packet-level Discrete-Event Network Simulation
- Open Access
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 1060-1073
Cowbird: Freeing CPUs to Compute by Offloading the Disaggregation of Memory
Memory disaggregation allows applications running on compute servers to expand their pool of available memory capacity by leveraging remote resources through low-latency networks. Unfortunately, in existing software-level disaggregation frameworks, the simple act of issuing requests to remote memory---paid on every access---can consume many CPU cycles. This overhead represents a direct cost to disaggregation, not only on the throughput of remote memory access but also on application logic, which must contend with the framework's CPU overheads. In this paper, we present Cowbird, a memory disaggregation architecture that frees compute servers to fulfill their stated purpose by removing disaggregation-related logic from their CPUs. Our experimental evaluation shows that Cowbird eliminates dis-aggregation overhead on compute-server CPUs and can improve end-to-end application performance by up to 3.5× compared to RDMA-only communication.
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 1060-1073
Open AccessCowbird: Freeing CPUs to Compute by Offloading the Disaggregation of Memory
Memory disaggregation allows applications running on compute servers to expand their pool of available memory capacity by leveraging remote resources through low-latency networks. Unfortunately, in existing software-level disaggregation frameworks, the simple act of issuing requests to remote memory---paid on every access---can consume many CPU cycles. This overhead represents a direct cost to disaggregation, not only on the throughput of remote memory access but also on application logic, which must contend with the framework's CPU overheads. In this paper, we present Cowbird, a memory disaggregation architecture that frees compute servers to fulfill their stated purpose by removing disaggregation-related logic from their CPUs. Our experimental evaluation shows that Cowbird eliminates dis-aggregation overhead on compute-server CPUs and can improve end-to-end application performance by up to 3.5× compared to RDMA-only communication.
- Open Access
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 167-181
DONS: Fast and Affordable Discrete Event Network Simulation with Automatic Parallelization
- 10 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 167-181
Open AccessDONS: Fast and Affordable Discrete Event Network Simulation with Automatic Parallelization