Bojie Lv
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Bojie Lv has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Rui Wang
Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China
11 shared publications - Tong Zhang
Nanjing University of Aeronautics and Astronautics
3 shared publications - Francis C. M. Lau
The University of Hong Kong
2 shared publications - Haisheng Tan
School of Computer Science and Technology, University of Science and Technology of China, Hefei, China
2 shared publications - Pei Xiao
University of Surrey
2 shared publications - Qianren Li
Department of Electronic and Electrical Engineering (EEE), Southern University of Science and Technology (SUSTech), China
2 shared publications
Latest Publications
- 8 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 25 (15361276) , 18185-18198
mmAlert: A Simultaneous Device Localization and Target Tracking System via Cooperative Passive Sensing
In this paper, a cooperative passive sensing system in millimeter-wave (mmWave) band for simultaneous device localization and target tracking, namely mmAlert, is proposed. Specifically, in uplink communication with at least two transmitters, the receiver receives the line-of-sight (LoS) signals and the scattered signals off a moving target, respectively. Based on the received signals of the sensing time intervals, when a passive target moves along one or multiple unknown trajectories, mmAlert could measure the angles-of-arrival (AoAs) and bistatic Doppler frequencies of the echoes from the sensing target, and then jointly estimate the locations of the transmitters and the trajectories of the target. Specifically, the transmitters’ locations and the moving target’s trajectories can be searched by minimizing the weighted mean squared error of the AoA and Doppler measurements. The optimal solution of the minimization problem is prohibitive due to the large number of variables. Hence, a low-complexity algorithm based on the alternating optimization is proposed, where the extended Kalman filter (EKF) is introduced to quickly shape the trajectories. The mmAlert is implemented in a 60GHz communication testbed. The experiment shows with the received signal spanning a single trajectory, the average localization error of the transmitters and average trajectory reconstruction error are 0.76 m and 0.29 m, respectively. The average errors are suppressed to 0.07 m and 0.2 m respectively, if the received signal spanning 50 trajectories is used. This justifies the benefit of trajectory diversity in localization and tracking.
- 8 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 25 (15361276) , 18185-18198
mmAlert: A Simultaneous Device Localization and Target Tracking System via Cooperative Passive Sensing
In this paper, a cooperative passive sensing system in millimeter-wave (mmWave) band for simultaneous device localization and target tracking, namely mmAlert, is proposed. Specifically, in uplink communication with at least two transmitters, the receiver receives the line-of-sight (LoS) signals and the scattered signals off a moving target, respectively. Based on the received signals of the sensing time intervals, when a passive target moves along one or multiple unknown trajectories, mmAlert could measure the angles-of-arrival (AoAs) and bistatic Doppler frequencies of the echoes from the sensing target, and then jointly estimate the locations of the transmitters and the trajectories of the target. Specifically, the transmitters’ locations and the moving target’s trajectories can be searched by minimizing the weighted mean squared error of the AoA and Doppler measurements. The optimal solution of the minimization problem is prohibitive due to the large number of variables. Hence, a low-complexity algorithm based on the alternating optimization is proposed, where the extended Kalman filter (EKF) is introduced to quickly shape the trajectories. The mmAlert is implemented in a 60GHz communication testbed. The experiment shows with the received signal spanning a single trajectory, the average localization error of the transmitters and average trajectory reconstruction error are 0.76 m and 0.29 m, respectively. The average errors are suppressed to 0.07 m and 0.2 m respectively, if the received signal spanning 50 trajectories is used. This justifies the benefit of trajectory diversity in localization and tracking.
- 22 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. PP (00189545) , 1-15
Mobility-Aware Adaptive Channel Contention for Delay-Sensitive Communications
Distributed channel access mechanisms such as carrier sense multiple access (CSMA), adopted in Wi-Fi networks, lack centralized coordination and global queue state information. Thus, they are inefficient in supporting delay-sensitive mobile traffic. For this problem, we introduce an adaptive channel contention mechanism aimed at improving queuing performance in a distributed millimeter-wave (mmWave) uplink system equipped with environment and mobility sensing capabilities. Specifically, mobile agents adjust their back-off timer parameters based on locally available information, including uplink queue lengths, current channel quality, and predicted channel statistics, where predictions are enabled by environmental and mobility sensing. We formulate the queuing optimization problem under this adaptive mechanism as a decentralized multi-agent Markov decision process (Dec-MA-MDP). Although contention decisions are made locally by each mobile agent, the corresponding policies for all agents are optimized centrally using system-wide statistics obtained prior to scheduling. In the proposed method, local policies are approximated via analytical models, reducing the policy optimization to a stochastic optimization problem defined along an adaptive Markov chain. To enhance efficiency, we further propose an unbiased gradient estimator, enabling policy refinement through stochastic gradient descent. Simulation results for both indoor and outdoor scenarios show that the proposed method achieves significantly higher optimization efficiency than conventional methods, such as simultaneous perturbation stochastic approximation (SPSA).
- 22 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. PP (00189545) , 1-15
Mobility-Aware Adaptive Channel Contention for Delay-Sensitive Communications
Distributed channel access mechanisms such as carrier sense multiple access (CSMA), adopted in Wi-Fi networks, lack centralized coordination and global queue state information. Thus, they are inefficient in supporting delay-sensitive mobile traffic. For this problem, we introduce an adaptive channel contention mechanism aimed at improving queuing performance in a distributed millimeter-wave (mmWave) uplink system equipped with environment and mobility sensing capabilities. Specifically, mobile agents adjust their back-off timer parameters based on locally available information, including uplink queue lengths, current channel quality, and predicted channel statistics, where predictions are enabled by environmental and mobility sensing. We formulate the queuing optimization problem under this adaptive mechanism as a decentralized multi-agent Markov decision process (Dec-MA-MDP). Although contention decisions are made locally by each mobile agent, the corresponding policies for all agents are optimized centrally using system-wide statistics obtained prior to scheduling. In the proposed method, local policies are approximated via analytical models, reducing the policy optimization to a stochastic optimization problem defined along an adaptive Markov chain. To enhance efficiency, we further propose an unbiased gradient estimator, enabling policy refinement through stochastic gradient descent. Simulation results for both indoor and outdoor scenarios show that the proposed method achieves significantly higher optimization efficiency than conventional methods, such as simultaneous perturbation stochastic approximation (SPSA).
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 73 (12) , 14048-14062
A Dynamic Programming Framework for Vehicular Task Offloading With Successive Action Improvement
In this paper, task offloading from vehicles with random velocities is optimized via a novel dynamic programming framework. Particularly, in a vehicular network with multiple vehicles and base stations (BSs), computing tasks of vehicles are offloaded via BSs to an edge server. Due to the random velocities, the exact locations of vehicles versus time, namely trajectories, cannot be determined in advance. Hence, instead of deterministic optimization, the cell association, uplink time, and throughput allocation of multiple vehicles during a period of task offloading are formulated as a finite-horizon Markov decision process. In order to derive a low-complexity solution algorithm, a two-time-scale framework is proposed. The scheduling period is divided into super slots, each super slot is further divided into a number of time slots. At the beginning of each super slot, we first obtain a reference scheduling scheme of cell association, uplink time and throughput allocation via deterministic optimization, yielding an approximation of the optimal value function. Within the super slot, the actual scheduling action of each time slot is determined by making improvement to the approximate value function according to the system state. Due to the successive improvement framework, a non-trivial average cost upper bound could be derived. In the simulation, the random trajectories of vehicles are generated from a high-fidelity traffic simulator. It is shown that the performance gain of the proposed scheduling framework over the baselines is significant.
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 73 (12) , 14048-14062
A Dynamic Programming Framework for Vehicular Task Offloading With Successive Action Improvement
In this paper, task offloading from vehicles with random velocities is optimized via a novel dynamic programming framework. Particularly, in a vehicular network with multiple vehicles and base stations (BSs), computing tasks of vehicles are offloaded via BSs to an edge server. Due to the random velocities, the exact locations of vehicles versus time, namely trajectories, cannot be determined in advance. Hence, instead of deterministic optimization, the cell association, uplink time, and throughput allocation of multiple vehicles during a period of task offloading are formulated as a finite-horizon Markov decision process. In order to derive a low-complexity solution algorithm, a two-time-scale framework is proposed. The scheduling period is divided into super slots, each super slot is further divided into a number of time slots. At the beginning of each super slot, we first obtain a reference scheduling scheme of cell association, uplink time and throughput allocation via deterministic optimization, yielding an approximation of the optimal value function. Within the super slot, the actual scheduling action of each time slot is determined by making improvement to the approximate value function according to the system state. Due to the successive improvement framework, a non-trivial average cost upper bound could be derived. In the simulation, the random trajectories of vehicles are generated from a high-fidelity traffic simulator. It is shown that the performance gain of the proposed scheduling framework over the baselines is significant.
- Open Access
A Dynamic Programming Framework for Vehicular Task Offloading with Successive Action Improvement
In this paper, task offloading from vehicles with random velocities is optimized via a novel dynamic programming framework. Particularly, in a vehicular network with multiple vehicles and base stations (BSs), computing tasks of vehicles are offloaded via BSs to an edge server. Due to the random velocities, the exact locations of vehicles versus time, namely trajectories, cannot be determined in advance. Hence, instead of deterministic optimization, the cell association, uplink time, and throughput allocation of multiple vehicles during a period of task offloading are formulated as a finite-horizon Markov decision process. In order to derive a low-complexity solution algorithm, a two-time-scale framework is proposed. The scheduling period is divided into super slots, each super slot is further divided into a number of time slots. At the beginning of each super slot, we first obtain a reference scheduling scheme of cell association, uplink time and throughput allocation via deterministic optimization, yielding an approximation of the optimal value function. Within the super slot, the actual scheduling action of each time slot is determined by making improvement to the approximate value function according to the system state. Due to the successive improvement framework, a non-trivial average cost upper bound could be derived. In the simulation, the random trajectories of vehicles are generated from a high-fidelity traffic simulator. It is shown that the performance gain of the proposed scheduling framework over the baselines is significant.
Open AccessA Dynamic Programming Framework for Vehicular Task Offloading with Successive Action Improvement
In this paper, task offloading from vehicles with random velocities is optimized via a novel dynamic programming framework. Particularly, in a vehicular network with multiple vehicles and base stations (BSs), computing tasks of vehicles are offloaded via BSs to an edge server. Due to the random velocities, the exact locations of vehicles versus time, namely trajectories, cannot be determined in advance. Hence, instead of deterministic optimization, the cell association, uplink time, and throughput allocation of multiple vehicles during a period of task offloading are formulated as a finite-horizon Markov decision process. In order to derive a low-complexity solution algorithm, a two-time-scale framework is proposed. The scheduling period is divided into super slots, each super slot is further divided into a number of time slots. At the beginning of each super slot, we first obtain a reference scheduling scheme of cell association, uplink time and throughput allocation via deterministic optimization, yielding an approximation of the optimal value function. Within the super slot, the actual scheduling action of each time slot is determined by making improvement to the approximate value function according to the system state. Due to the successive improvement framework, a non-trivial average cost upper bound could be derived. In the simulation, the random trajectories of vehicles are generated from a high-fidelity traffic simulator. It is shown that the performance gain of the proposed scheduling framework over the baselines is significant.
- Open Access
- 22 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Sensor Networks
- Vol. 20 (6) , 1-27
COSMO: Dynamic Uploading Scheduling in mmWave-Based Sensor Networks with Mobile Blockers
Wireless sensor networks (WSNs) leveraging millimeter wave (mmWave) communication for bandwidth-demanding applications is considered in this article. Despite the large bandwidth, the delivery of delay-sensitive information collected by sensors may still face significant latency due to the vulnerability to intermittent link blockage. Hence, the guarantee of low age of information (AoI) in mmWave WSNs is not straightforward. In this article, the wireless sensing and dynamic programming techniques are jointly exploited to relieve the above issue. The former tracks the human blockers and predicts the chance of link blockage; the latter optimizes the transmission of multiple sensors based on the prediction. Particularly, the long-term optimization of sampling, uplink time and power allocation policies in a sensor network can be formulated as an infinite-horizon Markov decision process (MDP) with discounted cost, where the state transition probabilities can be predicted via wireless sensing. A novel low-complexity solution framework, namely COSMO, with a guaranteed performance in the worst case, is proposed. Simulations show that compared with heuristic benchmarks, benefiting from the prediction of the link blockage, COSMO can significantly suppress the average system cost, which consists of both AoI and energy consumption.
- 22 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Sensor Networks
- Vol. 20 (6) , 1-27
Open AccessCOSMO: Dynamic Uploading Scheduling in mmWave-Based Sensor Networks with Mobile Blockers
Wireless sensor networks (WSNs) leveraging millimeter wave (mmWave) communication for bandwidth-demanding applications is considered in this article. Despite the large bandwidth, the delivery of delay-sensitive information collected by sensors may still face significant latency due to the vulnerability to intermittent link blockage. Hence, the guarantee of low age of information (AoI) in mmWave WSNs is not straightforward. In this article, the wireless sensing and dynamic programming techniques are jointly exploited to relieve the above issue. The former tracks the human blockers and predicts the chance of link blockage; the latter optimizes the transmission of multiple sensors based on the prediction. Particularly, the long-term optimization of sampling, uplink time and power allocation policies in a sensor network can be formulated as an infinite-horizon Markov decision process (MDP) with discounted cost, where the state transition probabilities can be predicted via wireless sensing. A novel low-complexity solution framework, namely COSMO, with a guaranteed performance in the worst case, is proposed. Simulations show that compared with heuristic benchmarks, benefiting from the prediction of the link blockage, COSMO can significantly suppress the average system cost, which consists of both AoI and energy consumption.
- 24 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 23 (10) , 13219-13234
Predictive Delay-Aware Scheduling With Receiver Rotation Detection and mmWave Channel Learning
In this paper, the joint downlink delay-aware scheduling in a large time span, where the rotation of User Equipments (UEs) may lead to significant channel variation, is investigated via a novel approximate Markov Decision Process (MDP) method. Specifically, we consider the joint downlink power allocation and receiving UE selection of a number of successive frames in a millimeter Wave (mmWave) system with quasi-static scattering clusters in the channel and rotating UEs. The propagation statistics of scattering clusters can be tracked via a learning method. Since the rotation of UEs can be detected, future channel statistics can be forecast via embedded motion sensors. Hence, the overall scheduling is formulated as a finite-horizon MDP with non-stationary predictable state transition probabilities, where the average queuing delay and probability of transmission buffer overflow are considered in the objective of scheduling optimization. A novel low-complexity solution framework with an analytical performance bound is proposed to save the efforts of value iteration. Benefiting from the forecast of system statistics, superior performance to the benchmarks is shown by numerical simulations, particularly in the suppression of buffer overflow rate. Preliminary experiments via an mmWave testbed are conducted to demonstrate the feasibility of the sensor-assisted mmWave beam alignment.
- 24 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 23 (10) , 13219-13234
Predictive Delay-Aware Scheduling With Receiver Rotation Detection and mmWave Channel Learning
In this paper, the joint downlink delay-aware scheduling in a large time span, where the rotation of User Equipments (UEs) may lead to significant channel variation, is investigated via a novel approximate Markov Decision Process (MDP) method. Specifically, we consider the joint downlink power allocation and receiving UE selection of a number of successive frames in a millimeter Wave (mmWave) system with quasi-static scattering clusters in the channel and rotating UEs. The propagation statistics of scattering clusters can be tracked via a learning method. Since the rotation of UEs can be detected, future channel statistics can be forecast via embedded motion sensors. Hence, the overall scheduling is formulated as a finite-horizon MDP with non-stationary predictable state transition probabilities, where the average queuing delay and probability of transmission buffer overflow are considered in the objective of scheduling optimization. A novel low-complexity solution framework with an analytical performance bound is proposed to save the efforts of value iteration. Benefiting from the forecast of system statistics, superior performance to the benchmarks is shown by numerical simulations, particularly in the suppression of buffer overflow rate. Preliminary experiments via an mmWave testbed are conducted to demonstrate the feasibility of the sensor-assisted mmWave beam alignment.
- 14 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 23 (9) , 10514-10529
Rate-Splitting With Hybrid Messages: DoF Analysis of the Two-User MIMO Broadcast Channel With Imperfect CSIT
Most of the existing research on degrees-of-freedom (DoF) with imperfect channel state information at the transmitter (CSIT) assume the messages are private, which may not reflect reality as the two receivers can request the same content. To overcome this limitation, we therefore consider the hybrid unicast and multicast messages. In particular, we characterize the optimal DoF region for the two-user multiple-input multiple-output (MIMO) broadcast channel (BC) with imperfect CSIT and hybrid messages. For the converse, we establish a three-step procedure to exploit the utmost possible relaxation. For the achievability, since the DoF region is with specific three-dimensional structure regarding antenna configurations and CSIT qualities, we verify the existence or non-existence of corner point candidates via the feature of antenna configurations and CSIT qualities categorization, and provide a hybrid message-aware rate-splitting scheme. Besides, we show that to achieve the strictly positive corner points, it is unnecessary to split the unicast messages into private and common parts. This implies adding a multicast message may mitigate the rate-splitting complexity.
- 14 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 23 (9) , 10514-10529
Rate-Splitting With Hybrid Messages: DoF Analysis of the Two-User MIMO Broadcast Channel With Imperfect CSIT
Most of the existing research on degrees-of-freedom (DoF) with imperfect channel state information at the transmitter (CSIT) assume the messages are private, which may not reflect reality as the two receivers can request the same content. To overcome this limitation, we therefore consider the hybrid unicast and multicast messages. In particular, we characterize the optimal DoF region for the two-user multiple-input multiple-output (MIMO) broadcast channel (BC) with imperfect CSIT and hybrid messages. For the converse, we establish a three-step procedure to exploit the utmost possible relaxation. For the achievability, since the DoF region is with specific three-dimensional structure regarding antenna configurations and CSIT qualities, we verify the existence or non-existence of corner point candidates via the feature of antenna configurations and CSIT qualities categorization, and provide a hybrid message-aware rate-splitting scheme. Besides, we show that to achieve the strictly positive corner points, it is unnecessary to split the unicast messages into private and common parts. This implies adding a multicast message may mitigate the rate-splitting complexity.
- Open Access
Rate-Splitting with Hybrid Messages: DoF Analysis of the Two-User MIMO Broadcast Channel with Imperfect CSIT
Most of the existing research on degrees-of-freedom (DoF) with imperfect channel state information at the transmitter (CSIT) assume the messages are private, which may not reflect reality as the two receivers can request the same content. To overcome this limitation, we therefore consider the hybrid unicast and multicast messages. In particular, we characterize the optimal DoF region for the two-user multiple-input multiple-output (MIMO) broadcast channel (BC) with imperfect CSIT and hybrid messages. For the converse, we establish a three-step procedure to exploit the utmost possible relaxation. For the achievability, since the DoF region is with specific three-dimensional structure regarding antenna configurations and CSIT qualities, we verify the existence or non-existence of corner point candidates via the feature of antenna configurations and CSIT qualities categorization and provide a hybrid message-aware rate-splitting scheme. Besides, we show that to achieve the strictly positive corner points, it is unnecessary to split the unicast messages into private and common parts. This implies adding a multicast message may mitigate the rate-splitting complexity.
Open AccessRate-Splitting with Hybrid Messages: DoF Analysis of the Two-User MIMO Broadcast Channel with Imperfect CSIT
Most of the existing research on degrees-of-freedom (DoF) with imperfect channel state information at the transmitter (CSIT) assume the messages are private, which may not reflect reality as the two receivers can request the same content. To overcome this limitation, we therefore consider the hybrid unicast and multicast messages. In particular, we characterize the optimal DoF region for the two-user multiple-input multiple-output (MIMO) broadcast channel (BC) with imperfect CSIT and hybrid messages. For the converse, we establish a three-step procedure to exploit the utmost possible relaxation. For the achievability, since the DoF region is with specific three-dimensional structure regarding antenna configurations and CSIT qualities, we verify the existence or non-existence of corner point candidates via the feature of antenna configurations and CSIT qualities categorization and provide a hybrid message-aware rate-splitting scheme. Besides, we show that to achieve the strictly positive corner points, it is unnecessary to split the unicast messages into private and common parts. This implies adding a multicast message may mitigate the rate-splitting complexity.
- 14 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 72 (7) , 4026-4040
Delay-Aware Two-Time-Scale Scheduling for mmWave Systems With Mobility and Environment Knowledge
A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.
- 14 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 72 (7) , 4026-4040
Delay-Aware Two-Time-Scale Scheduling for mmWave Systems With Mobility and Environment Knowledge
A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.
- 27 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 69 (7) , 7735-7750
Scheduling for Mobile Edge Computing With Random User Arrivals—An Approximate MDP and Reinforcement Learning Approach
In this paper, we investigate the scheduling design of a mobile edge computing (MEC) system, where active mobile devices with computation tasks randomly appear in a cell. Every task can be computed at either the mobile device or the MEC server. We jointly optimize the task offloading decision, uplink transmission device selection and power allocation by formulating the problem as an infinite-horizon Markov decision process (MDP). Compared with most of the existing literature, this is the first attempt to address the transmission and computation optimization with the random device arrivals in an infinite time horizon to our best knowledge. Due to the uncertainty in the device number and location, the conventional approximate MDP approaches addressing the curse of dimensionality cannot be applied. An alternative and suitable low-complexity solution framework is proposed in this work. We first introduce a baseline scheduling policy, whose value function can be derived analytically with the statistics of random mobile device arrivals. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. The complexity of deriving the sub-optimal policy is reduced dramatically compared with conventional solutions of MDP by eliminating the complicated value iteration. To address a more general scenario where the statistics of random mobile device arrivals are unknown, a novel and efficient algorithm integrating reinforcement learning and stochastic gradient descent (SGD) is proposed to improve the system performance in an online manner. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant.
- 27 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 69 (7) , 7735-7750
Scheduling for Mobile Edge Computing With Random User Arrivals—An Approximate MDP and Reinforcement Learning Approach
In this paper, we investigate the scheduling design of a mobile edge computing (MEC) system, where active mobile devices with computation tasks randomly appear in a cell. Every task can be computed at either the mobile device or the MEC server. We jointly optimize the task offloading decision, uplink transmission device selection and power allocation by formulating the problem as an infinite-horizon Markov decision process (MDP). Compared with most of the existing literature, this is the first attempt to address the transmission and computation optimization with the random device arrivals in an infinite time horizon to our best knowledge. Due to the uncertainty in the device number and location, the conventional approximate MDP approaches addressing the curse of dimensionality cannot be applied. An alternative and suitable low-complexity solution framework is proposed in this work. We first introduce a baseline scheduling policy, whose value function can be derived analytically with the statistics of random mobile device arrivals. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. The complexity of deriving the sub-optimal policy is reduced dramatically compared with conventional solutions of MDP by eliminating the complicated value iteration. To address a more general scenario where the statistics of random mobile device arrivals are unknown, a novel and efficient algorithm integrating reinforcement learning and stochastic gradient descent (SGD) is proposed to improve the system performance in an online manner. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant.