Xiaoyu Qiu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Xiaoyu Qiu has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
- Wuhui Chen
School of Software Engineering, Sun Yat-sen University, Guangzhou, China
9 shared publications - Zibin Zheng
Sun Yat-Sen University, Guangzhou, China
6 shared publications - Zicong Hong
Hong Kong University of Science and Technology, Hong Kong, Hong Kong
4 shared publications - Song Guo
Hong Kong University of Science and Technology, Hong Kong, China
3 shared publications - Bingxin Tang
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
2 shared publications - Hong-Ning Dai
Hong Kong Baptist University
2 shared publications
Latest Publications
- 27 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 23 (2) , 2257-2274
Scaling Blockchain via Dynamic Sharding
Sharding is considered a promising solution for scaling blockchain systems. However, most existing sharding systems have not considered the dynamics of the environment when making a sharding strategy, including the change of pending transactions, the leaving and joining of participants, and malicious attacks, which could cause performance instability and security issues. To address it, in this paper, we propose an intelligent and efficient dynamic sharding technology to advance the blockchain system performance and security. We first propose a formal and general evaluation framework for blockchain sharding in a dynamic environment, and conclude an optimization target for the system performance and security. To achieve a long-term benefit for the optimization target, a deep reinforcement learning (DRL)-based sharding approach has been proposed to intelligently make optimal sharding strategies. Next, we propose an adaptive resharding protocol to efficiently reduce the overhead introduced by dynamic sharding. Our experimental results illustrate that our proposed dynamic sharding in a simulation testbed can achieve 2.8 times transactions per second compared to traditional static sharding systems, and guarantee high security in a dynamic environment.
- 27 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 23 (2) , 2257-2274
Scaling Blockchain via Dynamic Sharding
Sharding is considered a promising solution for scaling blockchain systems. However, most existing sharding systems have not considered the dynamics of the environment when making a sharding strategy, including the change of pending transactions, the leaving and joining of participants, and malicious attacks, which could cause performance instability and security issues. To address it, in this paper, we propose an intelligent and efficient dynamic sharding technology to advance the blockchain system performance and security. We first propose a formal and general evaluation framework for blockchain sharding in a dynamic environment, and conclude an optimization target for the system performance and security. To achieve a long-term benefit for the optimization target, a deep reinforcement learning (DRL)-based sharding approach has been proposed to intelligently make optimal sharding strategies. Next, we propose an adaptive resharding protocol to efficiently reduce the overhead introduced by dynamic sharding. Our experimental results illustrate that our proposed dynamic sharding in a simulation testbed can achieve 2.8 times transactions per second compared to traditional static sharding systems, and guarantee high security in a dynamic environment.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 23 (6) , 7066-7083
Graph Neural Network-Enhanced Reinforcement Learning for Payment Channel Rebalancing
Building on top of blockchain, payment channel networks-backed (PCNs) cryptocurrencies emerge as a promising solution for a mobile payment system with fewer intermediaries, more security, higher speed, and lower cost. A key problem for PCN is payment channel rebalancing, that is, finding a set of circular transactions that restore a PCN with skewed channel balances back into an equilibrium state. However, existing practice on payment channel rebalancing either has a hard limit on the problem size or tends to fall into local optimum. To address these challenges, we propose DRL-PCR, a D eep R einforcement L earning-based P ayment C hannel R ebalancing algorithm. On one hand, DRL-PCR leverages the strong approximation ability of deep neural networks to handle large problem spaces. On the other hand, DRL-PCR decomposes the rebalancing problem into a sequence of decision-making problems and progressively builds the final solution. By aiming to find a globally optimized solution and solving the long-term optimization model of DRL, DRL-PCR is superior to greedy-based algorithms and can mitigate the risk of getting trapped in a local optimum. In particular, payment channel rebalancing typically involves dealing with graph-structured data, where the major obstacle lies in understanding the sophisticated circular dependencies between payment channels and routing paths. DRL-PCR achieves this by encoding the input data with a novel graph neural network-based model and capturing the circular dependencies through a customized message passing process. In addition, considering the distributed nature of PCN, DRL-PCR uses a leadership election protocol to elect leaders for decision-making. Evaluations on the historical data of two real-world PCNs demonstrate that DRL-PCR can restore the PCN to a more balanced state and improve the transaction throughput and success ratios by up to 2.1x and 1.6x, respectively.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 23 (6) , 7066-7083
Graph Neural Network-Enhanced Reinforcement Learning for Payment Channel Rebalancing
Building on top of blockchain, payment channel networks-backed (PCNs) cryptocurrencies emerge as a promising solution for a mobile payment system with fewer intermediaries, more security, higher speed, and lower cost. A key problem for PCN is payment channel rebalancing, that is, finding a set of circular transactions that restore a PCN with skewed channel balances back into an equilibrium state. However, existing practice on payment channel rebalancing either has a hard limit on the problem size or tends to fall into local optimum. To address these challenges, we propose DRL-PCR, a D eep R einforcement L earning-based P ayment C hannel R ebalancing algorithm. On one hand, DRL-PCR leverages the strong approximation ability of deep neural networks to handle large problem spaces. On the other hand, DRL-PCR decomposes the rebalancing problem into a sequence of decision-making problems and progressively builds the final solution. By aiming to find a globally optimized solution and solving the long-term optimization model of DRL, DRL-PCR is superior to greedy-based algorithms and can mitigate the risk of getting trapped in a local optimum. In particular, payment channel rebalancing typically involves dealing with graph-structured data, where the major obstacle lies in understanding the sophisticated circular dependencies between payment channels and routing paths. DRL-PCR achieves this by encoding the input data with a novel graph neural network-based model and capturing the circular dependencies through a customized message passing process. In addition, considering the distributed nature of PCN, DRL-PCR uses a leadership election protocol to elect leaders for decision-making. Evaluations on the historical data of two real-world PCNs demonstrate that DRL-PCR can restore the PCN to a more balanced state and improve the transaction throughput and success ratios by up to 2.1x and 1.6x, respectively.
- 6 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. 38 (4) , 166-173
Intelligence-Endogenous Management Platform for Computing and Network Convergence
Massive emerging applications are driving demand for the ubiquitous deployment of computing power today. This trend not only spurs the recent popularity of the Computing and Network Convergence (CNC), but also introduces an urgent need for the intelligentization of a management platform to coordinate changing resources and tasks in the CNC. Therefore, in this article, we present the concept of an intelligence-endogenous management platform for CNCs called CNC brain based on artificial intelligence technologies. It aims at efficiently and automatically matching the supply and demand with high heterogeneity in a CNC via four key building blocks, i.e., perception, scheduling, adaptation, and governance, throughout the CNC’s life cycle. Their functionalities, goals, and challenges are presented. To examine the effectiveness of the proposed concept and framework, we also implement a prototype for the CNC brain based on a deep reinforcement learning technology. Also, it is evaluated on a CNC testbed that integrates two open-source and popular frameworks (OpenFaas and Kubernetes) and a real-world business dataset provided by Microsoft Azure. The evaluation results prove the proposed method’s effectiveness in terms of resource utilization and performance. Finally, we highlight the future research directions of the CNC brain.
- 6 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. 38 (4) , 166-173
Intelligence-Endogenous Management Platform for Computing and Network Convergence
Massive emerging applications are driving demand for the ubiquitous deployment of computing power today. This trend not only spurs the recent popularity of the Computing and Network Convergence (CNC), but also introduces an urgent need for the intelligentization of a management platform to coordinate changing resources and tasks in the CNC. Therefore, in this article, we present the concept of an intelligence-endogenous management platform for CNCs called CNC brain based on artificial intelligence technologies. It aims at efficiently and automatically matching the supply and demand with high heterogeneity in a CNC via four key building blocks, i.e., perception, scheduling, adaptation, and governance, throughout the CNC’s life cycle. Their functionalities, goals, and challenges are presented. To examine the effectiveness of the proposed concept and framework, we also implement a prototype for the CNC brain based on a deep reinforcement learning technology. Also, it is evaluated on a CNC testbed that integrates two open-source and popular frameworks (OpenFaas and Kubernetes) and a real-world business dataset provided by Microsoft Azure. The evaluation results prove the proposed method’s effectiveness in terms of resource utilization and performance. Finally, we highlight the future research directions of the CNC brain.
- 27 February 2023
- Published by Springer Nature
- p. 193-221
Dynamic Sharding: A Trade-OFF Between Security and Scalability
- 27 February 2023
- Published by Springer Nature
- p. 193-221
Dynamic Sharding: A Trade-OFF Between Security and Scalability
- 25 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 20 (5) , 4372-4386
A Distributed and Privacy-Aware High-Throughput Transaction Scheduling Approach for Scaling Blockchain
Payment channel networks (PCNs) are considered as a prominent solution for scaling blockchain, where users can establish payment channels and complete transactions in an off-chain manner. However, it is non-trivial to schedule transactions in PCNs and most existing routing algorithms suffer from the following challenges: 1) one-shot optimization, 2) privacy-invasive channel probing, 3) vulnerability to DoS attacks. To address these challenges, we propose a privacy-aware transaction scheduling algorithm with defence against DoS attacks based on deep reinforcement learning (DRL), namely PTRD. Specifically, considering both the privacy preservation and long-term throughput into the optimization criteria, we formulate the transaction-scheduling problem as a Constrained Markov Decision Process. We then design PTRD, which extends off-the-shelf DRL algorithms to constrained optimization with an additional cost critic-network and an adaptive Lagrangian multiplier. Moreover, considering the distribution nature of PCNs, in which each user schedules transactions independently, we develop a distributed training framework to collect the knowledge learned by each agent so as to enhance learning effectiveness. With the customized network design and the distributed training framework, PTRD achieves a good balance between the optimization of the throughput and the minimization of privacy risks. Evaluations show that PTRD outperforms the state-of-the-art PCN routing algorithms by 2.7%–62.5% in terms of the long-term throughput while satisfying privacy constraints.
- 25 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 20 (5) , 4372-4386
A Distributed and Privacy-Aware High-Throughput Transaction Scheduling Approach for Scaling Blockchain
Payment channel networks (PCNs) are considered as a prominent solution for scaling blockchain, where users can establish payment channels and complete transactions in an off-chain manner. However, it is non-trivial to schedule transactions in PCNs and most existing routing algorithms suffer from the following challenges: 1) one-shot optimization, 2) privacy-invasive channel probing, 3) vulnerability to DoS attacks. To address these challenges, we propose a privacy-aware transaction scheduling algorithm with defence against DoS attacks based on deep reinforcement learning (DRL), namely PTRD. Specifically, considering both the privacy preservation and long-term throughput into the optimization criteria, we formulate the transaction-scheduling problem as a Constrained Markov Decision Process. We then design PTRD, which extends off-the-shelf DRL algorithms to constrained optimization with an additional cost critic-network and an adaptive Lagrangian multiplier. Moreover, considering the distribution nature of PCNs, in which each user schedules transactions independently, we develop a distributed training framework to collect the knowledge learned by each agent so as to enhance learning effectiveness. With the customized network design and the distributed training framework, PTRD achieves a good balance between the optimization of the throughput and the minimization of privacy risks. Evaluations show that PTRD outperforms the state-of-the-art PCN routing algorithms by 2.7%–62.5% in terms of the long-term throughput while satisfying privacy constraints.
- Top Cited Papers
- 13 April 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Surveys & Tutorials
- Vol. 23 (3) , 1659-1692
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
The incumbent Internet of Things suffers from poor scalability and elasticity exhibiting in communication, computing, caching and control (4Cs) problems. The recent advances in deep reinforcement learning (DRL) algorithms can potentially address the above problems of IoT systems. In this context, this paper provides a comprehensive survey that overviews DRL algorithms and discusses DRL-enabled IoT applications. In particular, we first briefly review the state-of-the-art DRL algorithms and present a comprehensive analysis on their advantages and challenges. We then discuss on applying DRL algorithms to a wide variety of IoT applications including smart grid, intelligent transportation systems, industrial IoT applications, mobile crowdsensing, and blockchain-empowered IoT. Meanwhile, the discussion of each IoT application domain is accompanied by an in-depth summary and comparison of DRL algorithms. Moreover, we highlight emerging challenges and outline future research directions in driving the further success of DRL in IoT applications.
- 13 April 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Surveys & Tutorials
- Vol. 23 (3) , 1659-1692
Top Cited PapersDeep Reinforcement Learning for Internet of Things: A Comprehensive Survey
The incumbent Internet of Things suffers from poor scalability and elasticity exhibiting in communication, computing, caching and control (4Cs) problems. The recent advances in deep reinforcement learning (DRL) algorithms can potentially address the above problems of IoT systems. In this context, this paper provides a comprehensive survey that overviews DRL algorithms and discusses DRL-enabled IoT applications. In particular, we first briefly review the state-of-the-art DRL algorithms and present a comprehensive analysis on their advantages and challenges. We then discuss on applying DRL algorithms to a wide variety of IoT applications including smart grid, intelligent transportation systems, industrial IoT applications, mobile crowdsensing, and blockchain-empowered IoT. Meanwhile, the discussion of each IoT application domain is accompanied by an in-depth summary and comparison of DRL algorithms. Moreover, we highlight emerging challenges and outline future research directions in driving the further success of DRL in IoT applications.
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 32 (5) , 1085-1101
Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective
Mobile edge computing (MEC) is a promising solution to support resource-constrained devices by offloading tasks to the edge servers. However, traditional approaches (e.g., linear programming and game-theory methods) for computation offloading mainly focus on the immediate performance, potentially leading to performance degradation in the long run. Recent breakthroughs regarding deep reinforcement learning (DRL) provide alternative methods, which focus on maximizing the cumulative reward. Nonetheless, there exists a large gap to deploy real DRL applications in MEC. This is because: 1) training a well-performed DRL agent typically requires data with large quantities and high diversity, 2) DRL training is usually accompanied by huge costs caused by trial-and-error. To address this mismatch, we study the applications of DRL on the multiuser computation offloading problem from a more practical perspective. In particular, we propose a distributed and collective DRL algorithm called DC-DRL with several improvements: 1) a distributed and collective training scheme that assimilates knowledge from multiple MEC environments, which not only greatly increases data amount and diversity but also spreads the exploration costs, 2) an updating method called adaptive n-step learning, which can improve training efficiency without suffering from the high variance caused by distributed training, 3) combining the advantages of deep neuroevolution and policy gradient to maximize the utilization of multiple environments and prevent the premature convergence. Lastly, evaluation results demonstrate the effectiveness of our proposed algorithm. Compared with the baselines, the exploration costs and final system costs are reduced by at least 43% and 9.4%, respectively.
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 32 (5) , 1085-1101
Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective
Mobile edge computing (MEC) is a promising solution to support resource-constrained devices by offloading tasks to the edge servers. However, traditional approaches (e.g., linear programming and game-theory methods) for computation offloading mainly focus on the immediate performance, potentially leading to performance degradation in the long run. Recent breakthroughs regarding deep reinforcement learning (DRL) provide alternative methods, which focus on maximizing the cumulative reward. Nonetheless, there exists a large gap to deploy real DRL applications in MEC. This is because: 1) training a well-performed DRL agent typically requires data with large quantities and high diversity, 2) DRL training is usually accompanied by huge costs caused by trial-and-error. To address this mismatch, we study the applications of DRL on the multiuser computation offloading problem from a more practical perspective. In particular, we propose a distributed and collective DRL algorithm called DC-DRL with several improvements: 1) a distributed and collective training scheme that assimilates knowledge from multiple MEC environments, which not only greatly increases data amount and diversity but also spreads the exploration costs, 2) an updating method called adaptive n-step learning, which can improve training efficiency without suffering from the high variance caused by distributed training, 3) combining the advantages of deep neuroevolution and policy gradient to maximize the utilization of multiple environments and prevent the premature convergence. Lastly, evaluation results demonstrate the effectiveness of our proposed algorithm. Compared with the baselines, the exploration costs and final system costs are reduced by at least 43% and 9.4%, respectively.
- 26 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 6 (6) , 9748-9761
Optimal Pricing Mechanism for Data Market in Blockchain-Enhanced Internet of Things
With the rapid development of the Internet of Things (IoT) in the era of big data, the amount of collected data has increased dramatically. Data are one of the most important commodities in IoT. To maximize the utility of the collected data, it is crucial to design an open IoT data market that enables data owners and consumers to carry out data trading securely and efficiently. To address the challenge of security presented by an untrusted and nontransparent data market, we propose an edge/cloud-computing-assisted, blockchain-enhanced data market framework to support secure and efficient IoT data trading, with a particular focus on an optimal pricing mechanism. In this mechanism, an authorized market-agency works as a scheduler, determining the win owner and its pricing strategy to the consumer. We formulate a two-stage Stackelberg game to solve the pricing and purchasing problem of the data consumer and the market-agency. In the first stage of the game, the market-agency gives the win owner and its pricing strategy. In the second stage, the data consumer decides on its purchasing quantity of data. We consider competition between data owners and propose a competition-enhanced pricing scheme. We apply backward induction to analyze the subgame perfect equilibrium at each stage for both independent and competition-enhanced pricing schemes. Lastly, we validate the existence and uniqueness of Stackelberg equilibrium, and the numerical results show the efficiency of the competition-enhanced pricing scheme.
- 26 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 6 (6) , 9748-9761
Optimal Pricing Mechanism for Data Market in Blockchain-Enhanced Internet of Things
With the rapid development of the Internet of Things (IoT) in the era of big data, the amount of collected data has increased dramatically. Data are one of the most important commodities in IoT. To maximize the utility of the collected data, it is crucial to design an open IoT data market that enables data owners and consumers to carry out data trading securely and efficiently. To address the challenge of security presented by an untrusted and nontransparent data market, we propose an edge/cloud-computing-assisted, blockchain-enhanced data market framework to support secure and efficient IoT data trading, with a particular focus on an optimal pricing mechanism. In this mechanism, an authorized market-agency works as a scheduler, determining the win owner and its pricing strategy to the consumer. We formulate a two-stage Stackelberg game to solve the pricing and purchasing problem of the data consumer and the market-agency. In the first stage of the game, the market-agency gives the win owner and its pricing strategy. In the second stage, the data consumer decides on its purchasing quantity of data. We consider competition between data owners and propose a competition-enhanced pricing scheme. We apply backward induction to analyze the subgame perfect equilibrium at each stage for both independent and competition-enhanced pricing schemes. Lastly, we validate the existence and uniqueness of Stackelberg equilibrium, and the numerical results show the efficiency of the competition-enhanced pricing scheme.
- Top Cited Papers
- 20 June 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 68 (8) , 8050-8062
Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing
Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize long-term offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Lastly, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies.
- 20 June 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 68 (8) , 8050-8062
Top Cited PapersOnline Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing
Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize long-term offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Lastly, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies.