Optimizing Microservice Deployment in Edge Computing with Large Language Models: Integrating Retrieval Augmented Generation and Chain of Thought Techniques
Open Access
Abstract
Large Language Models (LLMs) have demonstrated impressive capabilities in autogenerating code based on natural language instructions provided by humans. We observed that in the microservice models of edge computing, the problem of deployment latency optimization can be transformed into an NP-hard mathematical optimization problem. However, in the real world, deployment strategies at the edge often require immediate updates, while human-engineered code tends to be lagging. To bridge this gap, we innovatively integrated LLMs into the decision-making process for microservice deployment. Initially, we constructed a private Retrieval Augmented Generation (RAG) database containing prior knowledge. Subsequently, we employed meticulously designed step-by-step inductive instructions and used the chain of thought (CoT) technique to enable the LLM to learn, reason, reflect, and regenerate. We decomposed the microservice deployment latency optimization problem into a collection of granular sub-problems (described in natural language), progressively providing instructions to the fine-tuned LLM to generate corresponding code blocks. The generated code blocks underwent integration and consistency assessment. Additionally, we prompted the LLM to generate code without the use of the RAG database for comparative analysis. We executed the aforementioned code and comparison algorithm under identical operational environments and simulation parameters, conducting rigorous result analysis. Our fine-tuned model significantly reduced latencies by 22.8% in handling surges in request flows, 37.8% in managing complex microservice types, and 39.5% in processing increased network nodes compared to traditional algorithms. Moreover, our approach demonstrated marked improvements in latency performance over LLMs not utilizing RAG technology and reinforcement learning algorithms reported in other literature. The use of LLMs also highlights the concept of symmetry, as the symmetrical structure of input-output relationships in microservice deployment models aligns with the LLM’s inherent ability to process and generate balanced and optimized code. Symmetry in this context allows for more efficient resource allocation and reduces redundant operations, further enhancing the model’s effectiveness. We believe that LLMs hold substantial potential in optimizing microservice deployment models.Keywords
Funding Information
- Key Research and Development Program of Hubei Province (2022BAA038, 2023BAB074, 2024BAB031, 2022010702040061)
- Wuhan Artificial Intelligence Innovation (2022BAA038, 2023BAB074, 2024BAB031, 2022010702040061)
This publication has 37 references indexed in Scilit:
- Collaborative Deployment and Routing of Industrial Microservices in Smart FactoriesIEEE Transactions on Industrial Informatics, 2024
- On the Joint Design of Microservice Deployment and Routing in Cloud Data CentersJournal of Grid Computing, 2024
- PerfSim: A Performance Simulator for Cloud Native Microservice ChainsIEEE Transactions on Cloud Computing, 2021
- GPT-3: Its Nature, Scope, Limits, and ConsequencesMinds and Machines, 2020
- A Service Function Chain Deployment Method Based on Network Flow Theory for Load Balance in Operator NetworksIEEE Access, 2020
- The pains and gains of microservices: A Systematic grey literature reviewJournal of Systems and Software, 2018
- Deep Reinforcement Learning: A Brief SurveyIEEE Signal Processing Magazine, 2017
- A Survey on Mobile Edge Computing: The Communication PerspectiveIEEE Communications Surveys & Tutorials, 2017
- Adaptive Scheduling of Task Graphs with Dynamic ResilienceIEEE Transactions on Computers, 2016
- Virtual Servers Co-Migration for Mobile Accesses: Online versus Off-LineIEEE Transactions on Mobile Computing, 2015