Íñigo Goiri
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Íñigo Goiri has a total of 79 co-authors. Here are the top co-authors by number of shared publications:
- Ricardo Bianchini
Microsoft Corporation, Redmond, WA, USA
8 shared publications - Esha Choukse
Microsoft, Redmond, USA
7 shared publications - Rodrigo Fonseca
Microsoft Corporation, Redmond, WA, USA
7 shared publications - Chetan Bansal
Microsoft, Redmond, Washington, USA
4 shared publications - Anjaly Parayil
Microsoft, Bangalore, India
3 shared publications - Chaojie Zhang
Microsoft Corporation, Redmond, WA, USA
3 shared publications
Latest Publications
- Open Access
- 17 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 224-236
No Request Left Behind: Tackling Heterogeneity in Long-Context LLM Inference with Medha
Deploying million-token Large Language Models (LLMs) is challenging because production workloads are highly heterogeneous, mixing short queries and long documents. This heterogeneity, combined with the quadratic complexity of attention, creates severe convoy effects where long-running requests stall short, interactive ones, degrading system responsiveness. We present Medha, a serving system that eliminates these convoys by introducing fine-grained, preemptive scheduling to LLM inference. Medha makes preemption practical with a co-designed set of mechanisms – including Adaptive Chunking and Stream Pipeline Parallelism – that overcome the perceived inefficiencies and scaling challenges of chunking. Additionally, we present a new parallelism strategy KV-Cache Parallelism to reduce the decode latency and afford interactivity despite very long context. These mechanisms are orchestrated by a Length-Aware Relative Slack (LARS) scheduler, a deadline- and heterogeneity-aware scheduling policy that prevents both the convoy effect and the starvation that plagues simpler policies. Under a heterogeneous workload, Medha improves throughput by 5.7 × while reducing median and 90th-percentile latency by 30 × and 174 ×, respectively, compared to state-of-the-art non-preemptive systems.
- 17 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 224-236
Open AccessNo Request Left Behind: Tackling Heterogeneity in Long-Context LLM Inference with Medha
Deploying million-token Large Language Models (LLMs) is challenging because production workloads are highly heterogeneous, mixing short queries and long documents. This heterogeneity, combined with the quadratic complexity of attention, creates severe convoy effects where long-running requests stall short, interactive ones, degrading system responsiveness. We present Medha, a serving system that eliminates these convoys by introducing fine-grained, preemptive scheduling to LLM inference. Medha makes preemption practical with a co-designed set of mechanisms – including Adaptive Chunking and Stream Pipeline Parallelism – that overcome the perceived inefficiencies and scaling challenges of chunking. Additionally, we present a new parallelism strategy KV-Cache Parallelism to reduce the decode latency and afford interactivity despite very long context. These mechanisms are orchestrated by a Length-Aware Relative Slack (LARS) scheduler, a deadline- and heterogeneity-aware scheduling policy that prevents both the convoy effect and the starvation that plagues simpler policies. Under a heterogeneous workload, Medha improves throughput by 5.7 × while reducing median and 90th-percentile latency by 30 × and 174 ×, respectively, compared to state-of-the-art non-preemptive systems.
- Open Access
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1212-1227
Unipool: Efficient Management of Pre-Provisioned Resources for Container Platforms
- 14 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1212-1227
Open AccessUnipool: Efficient Management of Pre-Provisioned Resources for Container Platforms
- Open Access
- 19 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 817-830
M od S erve : Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
- 19 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 817-830
Open AccessM od S erve : Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 2203-2215
Workload Intelligence: Workload-Aware IaaS abstraction for Cloud Efficiency
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 2203-2215
Open AccessWorkload Intelligence: Workload-Aware IaaS abstraction for Cloud Efficiency
- Open Access
- 11 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 98-105
Cloud abstractions for AI workloads
- 11 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 98-105
Open AccessCloud abstractions for AI workloads
- Top Cited Papers
- 5 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 45 (4) , 54-59
Splitwise: Efficient Generative LLM Inference Using Phase Splitting
Generative Large Language Model (LLM) applications are rapidly growing, leading to widespread deployment of expensive, power-hungry GPUs. Growing power demands of AI in the cloud industry has become a global problem [5]. Our analysis shows that LLM inference involves two distinct phases: a compute-intensive prefill phase and a memory-intensive decode phase, each with different resource needs. Running them together introduces inefficient scheduling. Furthermore, unlike prefill phase, the decode phase can run on lower-cost and lowerpower hardware. Building on these insights, we propose Splitwise, a scheduling technique that splits prefill and decode phases across different machines to achieve better throughput. Additionally, Splitwise allows phase-specific hardware optimization. By efficiently transferring request state between machines, Splitwise achieves up to 2.35× more throughput within the same power and cost budgets, or 1.4× higher throughput at 20% lower cost and same power.
- 5 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 45 (4) , 54-59
Top Cited PapersSplitwise: Efficient Generative LLM Inference Using Phase Splitting
Generative Large Language Model (LLM) applications are rapidly growing, leading to widespread deployment of expensive, power-hungry GPUs. Growing power demands of AI in the cloud industry has become a global problem [5]. Our analysis shows that LLM inference involves two distinct phases: a compute-intensive prefill phase and a memory-intensive decode phase, each with different resource needs. Running them together introduces inefficient scheduling. Furthermore, unlike prefill phase, the decode phase can run on lower-cost and lowerpower hardware. Building on these insights, we propose Splitwise, a scheduling technique that splits prefill and decode phases across different machines to achieve better throughput. Additionally, Splitwise allows phase-specific hardware optimization. By efficiently transferring request state between machines, Splitwise achieves up to 2.35× more throughput within the same power and cost budgets, or 1.4× higher throughput at 20% lower cost and same power.
- Open Access
- 14 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 218-224
Towards Resource-Efficient Compound AI Systems
- 14 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 218-224
Open AccessTowards Resource-Efficient Compound AI Systems
- Open Access
- 5 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 136-146
Towards Workload-aware Cloud Efficiency: A Large-scale Empirical Study of Cloud Workload Characteristics
- 5 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 136-146
Open AccessTowards Workload-aware Cloud Efficiency: A Large-scale Empirical Study of Cloud Workload Characteristics
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 19-30
Performance Aware LLM Load Balancer for Mixed Workloads
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 19-30
Open AccessPerformance Aware LLM Load Balancer for Mixed Workloads
- Top Cited PapersOpen Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 1266-1281
TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 1266-1281
Top Cited PapersOpen AccessTAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms