Lei Jiao

While distributed speculative decoding can offer efficient acceleration for Large Language Model (LLM) inference in cloud-edge environments, unleashing its full potential confronts significant challenges, including complex token draft-length management, uncertain prompt arrivals and system conditions, and joint edge battery orchestration under fluctuating grid-energy prices. To systematically address these challenges, we conduct an intensive mathematical and algorithmic study. We first formulate a long-term cost minimization problem that captures all the challenges. Then, we decompose this problem into two subproblems and solve them alternately as time goes in an online manner. Particularly, we design a learning-centric algorithmic framework which employs bandit learning to address the intractability and manage draft length selection in the first subproblem and applies online learning to address the online uncertainty and manage battery states and grid-energy use in the second subproblem, both without relying on future inputs. We further conduct rigorous theoretical analysis, proving sub-linear regrets against offline optimal objectives and asymptotically-diminishing violation of long-term constraints. Extensive evaluations with real-world LLM inference demonstrate that our approach substantially outperforms several state-of-the-art methods, with many additional advantages.

The rising demand for blockchain interoperability is accelerating advancements in cross-chain bridge technologies, which are crucial for a seamless information transfer in multi-blockchain ecosystems. Existing blockchain bridges are typically classified into two categories: custodial and non-custodial. Custodial bridges use a trusted third party for easier and faster transactions but depend on custodian trust, while non-custodial bridges enhance transparency and control with smart contracts but increased complexity and latency. Currently, no bridge design successfully combines the benefits of both while avoiding their drawbacks. This paper presents EquiLink, a semi-custodial bridge that combines the benefits of both custodial and non-custodial methods. EquiLink employs a smart contract, known as the EquiLink Service, to initiate cross-chain transfers. It then uses the EquiLink Network, a system composed of remote-attested Trusted Execution Environments (TEEs), to verify and issue these transfers between two blockchains. Any eligible participants validated through remote attestation can join the EquiLink Network and contribute to the bridge's functionality. Additionally, participants are regulated by an economic model, providing an extra layer of security through economic incentives. This semi-custodial bridge enhances transparency and control for users. Meanwhile, it mitigates the risks associated with centralized custody and decentralization. In the evaluation, EquiLink is resilient against both replay and physical attacks. Additionally, it operates efficiently, reducing transaction costs by 14.1% and latency by 18.9%.

In the pursuit of data privacy and reduced latency, the adoption of edge intelligence has surged. Meanwhile, the enormous increase in AI has resulted in significant energy consumption. Edge intelligence plays a crucial role in Energy Demand Response (EDR). However, existing edge intelligence falls short of meeting the demands of co-locating training and inference tasks while satisfying EDR. Specifically, the intertwinement between balancing energy consumption, system delay and model accuracy, and uncertain future inputs adds to the challenge of designing an online sustainable system for co-located training and inference tasks. To address these challenges, we propose a novel two-timescale system for co-locating training and inference EDR. Our approach satisfies EDR by strategically planning training schedules on macro-timescales and migrating inference requests between heterogeneous edges on micro-timescales while minimizing long-term cost. We introduce a novel online polynomial time algorithm that first breaks down the problem into two subproblems, which are subsequently solved using an online-learning-based fractional algorithm and a randomized roun ding algorithm, respectively. Rigorous analysis demonstrates that our approach achieves both sublinear dynamic regret and sublinear dynamic fit. Extensive trace-driven evaluations validate the practical superiority of our approach over multiple existing methods, highlighting its effectiveness in real-world scenarios.

Artificial Intelligence-Generated Content (AIGC) has been gaining significant traction for automatic generation of diverse content. Due to the GPU-intensive generation process and the high costs associated with purchasing and operating GPUs, users often prefer to submit requests to a nearby edge cloud, maintained by an AIGC cloud service provider. Efficiently scheduling AIGC requests in the edge cloud faces non-trivial challenges. First, AIGC requests emphasize the quality of generated content, yet conventional scheduling algorithms often overlook this aspect. Second, when the volume of incoming requests exceeds the capacity of the cloud, the AIGC service provider needs to select appropriate requests to execute, which is further complicated by the online arrival pattern of requests and the constraints imposed by request deadlines. Third, users dynamically submit multiple requests at different times. To manage costs, each user operates within a pre-allocated budget for a given time period. For the AIGC cloud service provider, it is highly non-trivial to identify valuable requests and judiciously balance different user budgets. To tackle the above challenges, we target the online AIGC request scheduling problem with the new objective of maximizing the overall content generation quality. We first conduct real experiments to establish the quality model between inference steps and the quality of generated content. Then, based on this quality model, we formulate the problem into an integer linear program, which is proven NP-hard. Under a primal-dual framework, we carefully design the update of multiple dual variables, to flexibly control the consumption of edge resources and user budgets. We rigorously analyze the performance of the proposed algorithm and prove a theoretical performance guarantee on its competitive ratio. Extensive real-world trace-driven experiments manifest that our proposed method improves the state-of-the-art by up to 25.3% in overall content generation quality.

Transfer learning leverages existing models to help train new models, rather than training the new models from scratch. Unfortunately, realizing transfer learning in distributed cloud-edge networks faces critical challenges such as online training, uncertain network environments, time-coupled control decisions, and the balance between resource consumption and model accuracy. In this paper, targeting classification tasks, we study the settings of both homogeneous and heterogeneous transfer learning in cloud-edge networks via orchestrating model placement, data dispatching, and inference aggregation. We formulate non-linear mixed-integer programs of long-term cost optimization over consecutive time slots, and design polynomial-time online algorithms by exploiting the real-time trade-off between preserving previous control decisions and applying new control decisions. Our approaches produce new models by combining the existing pre-trained offline models and the online models that are continuously updated based on the inference results of data samples arriving in streams. We rigorously prove that our approaches only incur the number of inference mistakes no greater than a constant times that of the single best model in hindsight, and achieve constant competitive ratios for the total cost. Evaluations have confirmed the superior performance of our approaches compared to other state-of-the-art methods upon real-world data traces, under text classification transfer learning tasks.