Xiao Zhang

Large language models (LLMs) have emerged as a powerful foundation for intelligent reasoning and decision-making, demonstrating substantial impact across a wide range of domains and applications. However, their massive parameter scales and substantial resource demands pose critical challenges for efficient inference on edge devices. These devices are inherently constrained by limited computational power and memory capacity, while bandwidth bottlenecks at the network edge further restrict distributed deployment and real-time responsiveness. Although existing research has explored lightweight optimization techniques to mitigate memory limitations, such approaches often incur significant degradation in model accuracy and performance. To address these challenges, we propose LIME, a collaborative system that enables lossless inference for large models across multiple memory-constrained edge devices under limited network bandwidth. LIME employs an interleaved pipeline parallelism in conjunction with model offloading to dynamically balance computation and communication. Furthermore, a fine-grained offline allocation scheduler and online memory adaptation strategy are introduced to enhance the device's computing and storage resources while minimizing inference latency. Extensive experiments demonstrate that LIME, deployed on four heterogeneous Nvidia Jetson edge devices for LLaMA3.3-70B-Instruct model inference, achieves 1.7$\times$ and 3.7$\times$ speedups over state-of-the-art baselines under sporadic and bursty request patterns respectively, without compromising model accuracy.

Implementing large language models (LLMs)-driven root cause analysis (RCA) in cloud-native systems has become a key topic of modern software operations and maintenance. However, existing LLM-based approaches face three key challenges: multi-modality input constraint, context window limitation, and dynamic dependence graph. To address these issues, we propose a tool-assisted LLM agent with multi-modality observation data for fine-grained RCA, namely TAMO, including multi-modality alignment tool, root cause localization tool, and fault types classification tool. In detail, TAMO unifies multi-modal observation data into time-aligned representations for cross-modal feature consistency. Based on the unified representations, TAMO then invokes its specialized root cause localization tool and fault types classification tool for further identifying root cause and fault type underlying system context. This approach overcomes the limitations of LLMs in processing real-time raw observational data and dynamic service dependencies, guiding the model to generate repair strategies that align with system context through structured prompt design. Experiments on two benchmark datasets demonstrate that TAMO outperforms state-of-the-art (SOTA) approaches with comparable performance.

The hierarchical structure of distributed bilevel optimization (DBO) makes it more vulnerable to backdoor attacks than single-level optimization. Existing defense methods primarily focus on single-level settings, overlooking the cross-level propagation and amplification effects of attack impact in DBO. In this paper, we propose a defense mechanism $\textsf {BGFBL}$ for distributed bilevel optimization, which implements defenses for both the inner and outer layers, respectively. The inner-level defense corrects client gradient updates, while the outer-level defense introduces gradient clipping before aggregation to mitigate the impact of malicious updates and restrict attack spread. To the best of our knowledge, $\textsf {BGFBL}$ is the first theoretically guaranteed algorithm for distributed bilevel learning against model poisoning attacks. We demonstrate that $\textsf {BGFBL}$ achieves an asymptotically optimal convergence rate of $\mathcal {O}\left ({{\frac {1}{\sqrt {nK}}}}\right)$ , where n is the number of clients and K is the global maximum iteration number. Extensive experimental results demonstrate the effectiveness of our approach in defending against model poisoning attacks. Our method achieves a $30\%-50\%$ improvement in misclassification confidence compared to baselines.