Dynamic Quality-Latency Aware Routing for LLM Inference in Wireless Edge-Device Networks
Authors:
- 10 August 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
The integration of wireless communications and Large Language Models (LLMs) is poised to unlock ubiquitous intelligent services, yet deploying them in wireless edge-device collaborative environments presents a critical trade-off between inference quality and end-to-end latency. A fundamental mismatch exists between task complexity and resource allocation: offloading simple queries invites prohibitive latency, while on-device models lack the capacity for demanding computations. To address this challenge, we propose a dynamic, quality-latency aware routing framework that orchestrates inference between a lightweight model on the mobile device and a powerful model on the edge server. Our framework employs two distinct cost models: for single-turn queries, it fuses a BERT-predicted semantic score with communication and computation overheads; for multi-turn dialogues, it further quantifies context-aware costs arising from model switching and KV-cache management. While maintaining full inference quality, extensive experiments demonstrate that our framework cuts average response latency by 5-15% and reduces large model invocations by 10-20% against competitive baselines on MMLU, GSM8K, and MT-Bench-101 benchmarks.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- WDMoE: Wireless Distributed Mixture of Experts for Large Language ModelsIEEE Transactions on Wireless Communications, 2026
- MixLLM: Dynamic Routing in Mixed Large Language Models2025
- EdgeShard: Efficient LLM Inference via Collaborative Edge ComputingIEEE Internet of Things Journal, 2024
- Hybrid SLM and LLM for Edge-Cloud Collaborative Inference2024
- CDDM: Channel Denoising Diffusion Models for Wireless Semantic CommunicationsIEEE Transactions on Wireless Communications, 2024
- MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues2024