Recursive Offloading for LLM Serving in Multi-tier Networks
Authors:
- Zhiyuan WuZWZhiyuan Wu
- Sheng SunSSSheng Sun
- Yuwei WangYWYuwei Wang
- Min LiuMLMin Liu
- Bo GaoBo Gao
- Jinda LuJLJinda Lu
- Zheming YangZYZheming Yang
- Tian WenTWTian Wen
Preprint
Abstract
Heterogeneous device-edge-cloud computing infrastructures have become widely adopted in telecommunication operators and Wide Area Networks (WANs), offering multi-tier computational support for emerging intelligent services. With the rapid proliferation of Large Language Model (LLM) services, efficiently coordinating inference tasks and reducing communication overhead within these multi-tier network architectures becomes a critical deployment challenge. Existing LLM serving paradigms exhibit significant limitations: on-device deployment supports only lightweight LLMs due to hardware constraints, while cloud-centric deployment suffers from resource congestion and considerable prompt communication overhead caused by frequent service requests during peak periods. Although the model-cascading-based inference strategy adapts better to multi-tier networks, its reliance on fine-grained, manually adjusted thresholds makes it less responsive to dynamic network conditions and varying task complexities. To address these challenges, we propose RecServe, a recursive offloading framework tailored for LLM serving in multi-tier networks. RecServe integrates a task-specific hierarchical confidence evaluation mechanism that guides offloading decisions based on inferred task complexity in progressively scaled LLMs across device, edge, and cloud tiers. To further enable intelligent task routing across tiers, RecServe employs a sliding-window-based dynamic offloading strategy with quantile interpolation, enabling real-time tracking of historical confidence distributions and adaptive offloading threshold adjustments. Experiments on eight datasets demonstrate that RecServe outperforms CasServe in both service quality and communication efficiency, and reduces the communication burden by over 50\% compared to centralized cloud-based serving.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Pushing Large Language Models to the 6G Edge: Vision, Challenges, and OpportunitiesIEEE Communications Magazine, 2025
- EdgeShard: Efficient LLM Inference via Collaborative Edge ComputingIEEE Internet of Things Journal, 2024
- Task offloading strategies for mobile edge computing: A surveyComputer Networks, 2024
- End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2024
- Hierarchical Federated Learning With Momentum Acceleration in Multi-Tier NetworksIEEE Transactions on Parallel and Distributed Systems, 2023
- Split Computing and Early Exiting for Deep Learning Applications: Survey and Research ChallengesACM Computing Surveys, 2022
- Distributed Artificial Intelligence Empowered by End-Edge-Cloud Computing: A SurveyIEEE Communications Surveys & Tutorials, 2022
- A Survey on Task Offloading in Multi-access Edge ComputingJournal of Systems Architecture, 2021
- Task offloading in Edge and Cloud Computing: A survey on mathematical, artificial intelligence and control theory solutionsComputer Networks, 2021
- Edge Computing: Vision and ChallengesIEEE Internet of Things Journal, 2016