OrionInfer: Low-Overhead Parallelism Switching and Live Migration for Efficient LLM Serving
Open Access
- 8 August 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1132-1142
Abstract
No abstract availableFunding Information
- National Natural Science Foundation of China (72595845, 7259584)
This publication has 16 references indexed in Scilit:
- PipeRAG: Fast Retrieval-Augmented Generation via Adaptive Pipeline Parallelism2025
- Industrial applications of large language modelsScientific Reports, 2025
- POD-A ttention : Unlocking Full Prefill-Decode Overlap for Faster LLM Inference2025
- CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion2025
- Splitwise: Efficient Generative LLM Inference Using Phase Splitting2024
- SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification2024
- SpotServe: Serving Generative Large Language Models on Preemptible Instances2024
- ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference2024
- A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and ChallengesIEEE Access, 2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023