MaaI: ML-as-an-Infrastructure for Diverse Personalized Inference With Edge Offloading
Open Access
- 6 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 24801-24811
Abstract
The rapid advancements in large language models (LLMs) have transformed applications across industries, from natural language understanding to content generation. However, their escalating computational demands and dependence on cloud services create barriers to the widespread adoption of customized models. We introduce MaaI, an edge-offloading architecture that optimizes LLM deployment by distributing computational tasks and customization across edge devices and cloud servers. Crucially, MaaI identifies the optimal partition point to divide the model between the server and edge, ensuring that foundational components are allocated to edge devices while intensive computations remain in the cloud. Utilizing quantization and LoRA adapters, this approach enables users to leverage larger models with enhanced performance while minimizing server load. Furthermore, by retaining only the foundational model on the server and offloading customized adaptations to edge devices, MaaI facilitates scalable, personalized deployments without significantly increasing server resource requirements. Experimental results across diverse benchmarks and hardware configurations confirm MaaI’s effectiveness in optimizing performance, memory efficiency, and latency.Keywords
Funding Information
- National Research Foundation of Korea (RS-2025-08182968)
This publication has 7 references indexed in Scilit:
- AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and AccelerationGetMobile: Mobile Computing and Communications, 2025
- Toward expert-level medical question answering with large language modelsNature Medicine, 2025
- Large language models encode clinical knowledgeNature, 2023
- What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical ExamsApplied Sciences, 2021
- The Power of Scale for Parameter-Efficient Prompt Tuning2021
- Delays in Human-Computer Interaction and Their Effects on Brain ActivityPLOS ONE, 2016
- Delays and User Performance in Human-Computer-Network Interaction TasksHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2009