Haitao Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Haitao Li has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
Changjun Liu
Faculty of Information Technology, University of Beijing University of Technology, Beijing 100124 China
3 shared publicationsXin Lv
Faculty of Information Technology, Beijing University of Technology, Beijing, China
3 shared publicationsJiawei Huang
Beijing Univ Technol, Sch Informat Sci & Technol, Beijing 100124, Peoples R China
2 shared publicationsJiawei Luo
Beijing University of Technology
2 shared publicationsShuai Zhang
Faculty of Information Technology, Beijing University of Technology, Beijing, China
2 shared publicationsYiming Qian
Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China
2 shared publications
Latest Publications
- 6 November 2024
- Published by Elsevier in Vehicular Communications
- Vol. 50
Hierarchical federated deep reinforcement learning based joint communication and computation for UAV situation awareness
- 6 November 2024
- Published by Elsevier in Vehicular Communications
- Vol. 50
Hierarchical federated deep reinforcement learning based joint communication and computation for UAV situation awareness
- 1 January 2024
- Published by Springer Nature in Wireless Personal Communications
- Vol. 134 (2) , 809-829
Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication
- 1 January 2024
- Published by Springer Nature in Wireless Personal Communications
- Vol. 134 (2) , 809-829
Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication
- 24 October 2023
- Published by Springer Nature in Wireless Networks
- Vol. 30 (2) , 923-937
Anti-jamming transmission in softwarization UAV network: a federated deep reinforcement learning approach
- 24 October 2023
- Published by Springer Nature in Wireless Networks
- Vol. 30 (2) , 923-937
Anti-jamming transmission in softwarization UAV network: a federated deep reinforcement learning approach
- Open Access
- 17 May 2022
- Published by Springer Nature in Research Square
Multiobjective Deep Reinforcement Learning based Joint Beamforming and Power Allocation in UAV assisted Cellular Communication
In order to provide spectrum and energy efficient communication for unmanned aerial vehicle (UAV) assisted cellular network, the problem of joint beamforming and power allocation (JBPA) in aerial multicell scenario is addressed. The JBPA multiobjective optimization model which would simultaneously maximize the achievable spectrum and energy efficiency is first developed. In view of the model, the centralized deep reinforcement learning (DRL) algorithm, i.e., upper confidence bound based Dueling deep Q network (UCB DDQN) with Mish activation function, is proposed to solve the multiobjective optimization problem and we make use of this learning algorithm to design joint beamforming and power allocation strategy. Furthermore, a federated UCB DDQN learning based JBPA is to proposed tackle the challenge of centralized DRL would require excessive data exchange. Simulation results validate that the faster convergence speed and the total weighted energy-spectrum efficiency (TWESE) achieved by the joint beamforming and power allocation based on UCB DDQN is greater than conventional DQN based resource allocation approach, and show the superior TWESE performance federated UCB DDQN achieve compared to centralized UCB DDQN.
- 17 May 2022
- Published by Springer Nature in Research Square
Open AccessMultiobjective Deep Reinforcement Learning based Joint Beamforming and Power Allocation in UAV assisted Cellular Communication
In order to provide spectrum and energy efficient communication for unmanned aerial vehicle (UAV) assisted cellular network, the problem of joint beamforming and power allocation (JBPA) in aerial multicell scenario is addressed. The JBPA multiobjective optimization model which would simultaneously maximize the achievable spectrum and energy efficiency is first developed. In view of the model, the centralized deep reinforcement learning (DRL) algorithm, i.e., upper confidence bound based Dueling deep Q network (UCB DDQN) with Mish activation function, is proposed to solve the multiobjective optimization problem and we make use of this learning algorithm to design joint beamforming and power allocation strategy. Furthermore, a federated UCB DDQN learning based JBPA is to proposed tackle the challenge of centralized DRL would require excessive data exchange. Simulation results validate that the faster convergence speed and the total weighted energy-spectrum efficiency (TWESE) achieved by the joint beamforming and power allocation based on UCB DDQN is greater than conventional DQN based resource allocation approach, and show the superior TWESE performance federated UCB DDQN achieve compared to centralized UCB DDQN.
- Open Access
- 23 March 2022
- Published by Frontiers Media SA in Frontiers in Energy Research
- Vol. 10
Microgrid Energy Management Strategy Base on UCB-A3C Learning
The uncertainty of renewable energy and demand response brings many challenges to the microgrid energy management. Driven by the recent advances and applications of deep reinforcement learning a microgrid energy management strategy, i.e., upper confidence bound based advantage actor-critic (A3C), is proposed to utilize a novel action exploration mechanism to learn the power output of wind power generation, the price of electricity trading and power load. The simulation results indicate that the UCB-A3C learning based energy management strategy is better than conventional PPO, actor critical and A3C algorithm.
- 23 March 2022
- Published by Frontiers Media SA in Frontiers in Energy Research
- Vol. 10
Open AccessMicrogrid Energy Management Strategy Base on UCB-A3C Learning
The uncertainty of renewable energy and demand response brings many challenges to the microgrid energy management. Driven by the recent advances and applications of deep reinforcement learning a microgrid energy management strategy, i.e., upper confidence bound based advantage actor-critic (A3C), is proposed to utilize a novel action exploration mechanism to learn the power output of wind power generation, the price of electricity trading and power load. The simulation results indicate that the UCB-A3C learning based energy management strategy is better than conventional PPO, actor critical and A3C algorithm.
- Open Access
- 1 April 2021
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1873 (1) , 012042
Research on QoS Centralized Routing of Software-Defined Aeronautic Swarm Network
To address the problem of unstable data transmission caused by frequent topology changes in Aeronautic Swarm Network (ASNET), a centralized routing algorithm for Quality of Service(QoS) of the aeronautic swarm network based on Software Defined Network(SDN) is studied. First, this paper designs a Software Defined Aeronautic Swarm Network (SD-ASNET) model. Then, taking advantage of the SDN controller’s ability to obtain global link-state information, we propose a Multiple Link-state Awareness routing (MLSA) based on the shortest path algorithm, taking into account link bandwidth, transmission delay and other factors. The simulation results show that the proposed centralized routing protocol makes full use of the centralized decision-making capability of the SDN controller, and effectively optimizes the link bandwidth utilization and transmission delay by using link parameter information.
- 1 April 2021
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1873 (1) , 012042
Open AccessResearch on QoS Centralized Routing of Software-Defined Aeronautic Swarm Network
To address the problem of unstable data transmission caused by frequent topology changes in Aeronautic Swarm Network (ASNET), a centralized routing algorithm for Quality of Service(QoS) of the aeronautic swarm network based on Software Defined Network(SDN) is studied. First, this paper designs a Software Defined Aeronautic Swarm Network (SD-ASNET) model. Then, taking advantage of the SDN controller’s ability to obtain global link-state information, we propose a Multiple Link-state Awareness routing (MLSA) based on the shortest path algorithm, taking into account link bandwidth, transmission delay and other factors. The simulation results show that the proposed centralized routing protocol makes full use of the centralized decision-making capability of the SDN controller, and effectively optimizes the link bandwidth utilization and transmission delay by using link parameter information.
- 23 November 2019
- Published by Springer Nature in Wireless Personal Communications
- Vol. 111 (4) , 2107-2127
Reinforcement Learning Based Full-Duplex Cognitive Anti-jamming Using Improved Energy Detector
- 23 November 2019
- Published by Springer Nature in Wireless Personal Communications
- Vol. 111 (4) , 2107-2127
Reinforcement Learning Based Full-Duplex Cognitive Anti-jamming Using Improved Energy Detector
- Open Access
- 1 February 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 30234-30243
Selfish Bandit-Based Cognitive Anti-Jamming Strategy for Aeronautic Swarm Network in Presence of Multiple Jammer
In order to enhance the anti-jamming capability of aeronautic swarm tactical network in complicated electromagnetic environment, we address the problem of bandit-based cognitive anti-jamming strategy for enabling reliable information transmission. We first present an adversarial multiuser multi-armed bandit model for aeronautic swarm network employing airborne cognitive radios with same-frequency simultaneous transmit and receive feature. Then, we utilize the improved energy detection method to perform jamming sensing and derive the closed expression of false alarm probability, false detection probability and the optimal decision threshold in the case of single and multi-jammer. Finally, using the jamming sensing output to calculate reward and with the objective of maximizing the throughput of each airborne radio, a decentralized selfish doubling trick (DT) kl-UCB++ anti-jamming strategy is developed to allocate an optimal configuration of transmit power and spectrum channel to each radio. This anytime bandit strategy is simultaneously minimax optimal and asymptotically optimal. Simulation results validate that the aggregate average throughput, cumulative regret obtained with the proposed anti-jamming strategy outperform the wellknown UCB kl-UCB++ bandit algorithm.
- 1 February 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 30234-30243
Open AccessSelfish Bandit-Based Cognitive Anti-Jamming Strategy for Aeronautic Swarm Network in Presence of Multiple Jammer
In order to enhance the anti-jamming capability of aeronautic swarm tactical network in complicated electromagnetic environment, we address the problem of bandit-based cognitive anti-jamming strategy for enabling reliable information transmission. We first present an adversarial multiuser multi-armed bandit model for aeronautic swarm network employing airborne cognitive radios with same-frequency simultaneous transmit and receive feature. Then, we utilize the improved energy detection method to perform jamming sensing and derive the closed expression of false alarm probability, false detection probability and the optimal decision threshold in the case of single and multi-jammer. Finally, using the jamming sensing output to calculate reward and with the objective of maximizing the throughput of each airborne radio, a decentralized selfish doubling trick (DT) kl-UCB++ anti-jamming strategy is developed to allocate an optimal configuration of transmit power and spectrum channel to each radio. This anytime bandit strategy is simultaneously minimax optimal and asymptotically optimal. Simulation results validate that the aggregate average throughput, cumulative regret obtained with the proposed anti-jamming strategy outperform the wellknown UCB kl-UCB++ bandit algorithm.
- 27 December 2018
- Published by Wiley in International Journal of Communication Systems
- Vol. 32 (5)
Cross‐layer optimization for full‐duplex cognitive radio network with cooperative spectrum sensing
Summary: In this paper, the cross‐layer optimal design of multihop ad hoc network employing full‐duplex cognitive radios (CRs) is investigated. Firstly, the analytical expressions of cooperative spectrum sensing performance for multihop CR networks over composite fading channels are derived. Then, the opportunistic throughput and transmit power of CRs are presented based on the derivation of false alarm and missed detection probability. Finally, a multiobjective optimization model is proposed to maximize the opportunistic throughputs and minimize the transmitting power. Simulation results indicate that Pareto optimal solution of sensing duration, decision threshold, and transmit power can be achieved by cross‐layer multiobjective optimization, it can balance the conflicts between different objective functions and reap the acceptable outcomes for multihop CR network.
- 27 December 2018
- Published by Wiley in International Journal of Communication Systems
- Vol. 32 (5)
Cross‐layer optimization for full‐duplex cognitive radio network with cooperative spectrum sensing
Summary: In this paper, the cross‐layer optimal design of multihop ad hoc network employing full‐duplex cognitive radios (CRs) is investigated. Firstly, the analytical expressions of cooperative spectrum sensing performance for multihop CR networks over composite fading channels are derived. Then, the opportunistic throughput and transmit power of CRs are presented based on the derivation of false alarm and missed detection probability. Finally, a multiobjective optimization model is proposed to maximize the opportunistic throughputs and minimize the transmitting power. Simulation results indicate that Pareto optimal solution of sensing duration, decision threshold, and transmit power can be achieved by cross‐layer multiobjective optimization, it can balance the conflicts between different objective functions and reap the acceptable outcomes for multihop CR network.