Abstract
The rapid development of artificial intelligence has promoted the application of AI services in various fields. The combination of the microservice architecture and the “AI as a Service” (AaaS) to form an intelligent microservice system can realize the orchestration of intelligent applications. In intelligent applications, AI services often require the key functions of accompanying microservices, such as service discovery, data collection, gateway routing, etc., to ensure the stability and efficiency. Otherwise, it will lead to the incompleteness of applications, severely affecting the overall performance of the system. Therefore, fine-grained hybrid orchestration of AI services and microservices is crucial. Nevertheless, it faces major challenges in the resource-constrained edge environment: complex dependencies, frequent data communication, and heterogeneous resource competition between microservices and AI services. Moreover, the tight coupling between deployment and routing leads to complex joint optimization problems. Considering either deployment or routing separately will lead to poor system performance. However, most of the existing studies ignore the above problems.To this end, this paper investigates the hybrid orchestration problem. First, we model intelligent applications as hybrid service invocation chains. Then, based on Jackson queuing networks and multi-instance models, we deeply study the complex dependencies among hybrid services and the heterogeneous resource allocation problem, and conduct an accurate analysis of the request delay and system energy consumption. Furthermore, we innovatively propose Lagrangian Relaxed Deep Deterministic Policy Gradient (LR_DDPG), which introduces a cost mechanism and transforms the constrained hybrid orchestration process into an unconstrained optimization problem by Lagrange relaxation to find the optimal deployment and routing strategy under complex constraints. Finally, the experimental results show that our algorithm achieves significant advantages in terms of latency, energy consumption, and request success rate.
Funding Information
  • Hubei Provincial Technology Innovation Plan (2024BAB016)
  • Three Gorges Hi-Tech Information Technology Co. (2025KYXM004)

This publication has 65 references indexed in Scilit: