Jaewoo So

Integrated sensing and communications (ISAC) is a promising key technology for vehicular networks, because it allows roadside units to support both data transmission and radar-like sensing over the same spectrum and hardware platform. In conventional time division ISAC systems, each frame is divided into sensing and communication phases with a fixed ratio, which determines the tradeoff between the sensing accuracy and the communication throughput. However, in high-mobility vehicular environments, a fixed sensing–communication split is often suboptimal due to time-varying channel and intervehicle interference variations. In this paper, we propose a dynamic sensing–communication time division and ISAC beamforming scheme that minimizes the Cramér–Rao lower bound while satisfying the minimum effective communication sum rate. We further develop a deep reinforcement learning framework based on proximal policy optimization to find the optimal time division ratio and beamforming vectors. Simulation results show that the proposed dynamic time division beamforming scheme significantly outperforms the conventional fixed time division beamforming schemes in terms of sensing accuracy and the communication sum rate.

In a multi-cell network, interference management between adjacent cells is a key factor that determines the performance of the entire cellular network. In particular, in order to control inter-cell interference while providing a high data rate to users, it is very important for the base station (BS) of each cell to appropriately control the transmit power in the downlink. However, as the number of cells increases, controlling the downlink transmit power at the BS becomes increasingly difficult. In this paper, we propose a multi-agent deep reinforcement learning (MADRL)-based transmit power control scheme to maximize the sum rate in multi-cell networks. In particular, the proposed scheme incorporates a long short-term memory (LSTM) architecture into the MADRL scheme to retain state information across time slots and to use that information for subsequent action decisions, thereby improving the sum rate performance. In the proposed scheme, the agent of each BS uses only its local channel state information; consequently, it does not need to receive signal messages from adjacent agents. The simulation results show that the proposed scheme outperforms the existing MADRL scheme by reducing the amount of signal messages exchanged between links and improving the sum rate.

In unmanned aerial vehicle (UAV)-mounted base station (MBS) networks, user equipment (UE) experiences dynamic channel variations because of the mobility of the UAV and the changing weather conditions. In order to overcome the degradation in the quality of service (QoS) of the UE due to channel variations, it is important to appropriately determine the three-dimensional (3D) position and transmission power of the base station (BS) mounted on the UAV. Moreover, it is also important to account for both geographical and meteorological factors when deploying UAV-MBSs because they service ground UE in various regions and atmospheric environments. In this paper, we propose an energy-efficient UAV-MBS deployment scheme in multi-UAV-MBS networks using a hybrid improved simulated annealing–particle swarm optimization (ISA-PSO) algorithm to find the 3D position and transmission power of each UAV-MBS. Moreover, we developed a simulator for deploying UAV-MBSs, which took the dynamic weather conditions into consideration. The proposed scheme for deploying UAV-MBSs demonstrated superior performance, where it achieved faster convergence and higher stability compared with conventional approaches, making it well suited for practical deployment. The developed simulator integrates terrain data based on geolocation and real-time weather information to produce more practical results.

The directional antenna combined with beamforming is one of the attractive solutions to accommodate high data rate applications in 5G vehicle communications. However, the directional nature of beamforming requires beam alignment between the transmitter and the receiver, which incurs significant signaling overhead. Hence, we need to find the optimal parameters for directional beamforming, i.e., the antenna beamwidth and beam alignment interval, that maximize the throughput, taking the beam alignment overhead into consideration. In this paper, we propose a reinforcement learning (RL)-based beamforming scheme in a vehicle-to-infrastructure system, where we jointly determine the antenna beamwidth and the beam alignment interval, taking into account the past and future rewards. The simulation results show that the proposed RL-based joint beamforming scheme outperforms conventional beamforming schemes in terms of the average throughput and the average link stability ratio.

As the number of Internet of things (IoT) devices increases exponentially, scheduling and managing the radio resources for IoT devices has become more important. To efficiently allocate radio resources, the base station (BS) needs the channel state information (CSI) of devices every time. Hence, each device needs to periodically (or aperiodically) report its channel quality indicator (CQI) to the BS. The BS determines the modulation and coding scheme (MCS) based on the CQI reported by the IoT device. However, the more a device reports its CQI, the more the feedback overhead increases. In this paper, we propose a long short-term memory (LSTM)-based CQI feedback scheme, where the IoT device aperiodically reports its CQI relying on an LSTM-based channel prediction. Additionally, because the memory capacity of IoT devices is generally small, the complexity of the machine learning model must be reduced. Hence, we propose a lightweight LSTM model to reduce the complexity. The simulation results show that the proposed lightweight LSTM-based CSI scheme dramatically reduces the feedback overhead compared with that of the existing periodic feedback scheme. Moreover, the proposed lightweight LSTM model significantly reduces the complexity without sacrificing performance.

Recently, with the development of autonomous driving technology, vehicle-to-everything (V2X) communication technology that provides a wireless connection between vehicles, pedestrians, and roadside base stations has gained significant attention. Vehicle-to-vehicle (V2V) communication should provide low-latency and highly reliable services through direct communication between vehicles, improving safety. In particular, as the number of vehicles increases, efficient radio resource management becomes more important. In this paper, we propose a deep reinforcement learning (DRL)-based decentralized resource allocation scheme in the V2X communication network in which the radio resources are shared between the V2V and vehicle-to-infrastructure (V2I) networks. Here, a deep Q-network (DQN) is utilized to find the resource blocks and transmit power of vehicles in the V2V network to maximize the sum rate of the V2I and V2V links while reducing the power consumption and latency of V2V links. The DQN also uses the channel state information, the signal-to-interference-plus-noise ratio (SINR) of V2I and V2V links, and the latency constraints of vehicles to find the optimal resource allocation scheme. The proposed DQN-based resource allocation scheme ensures energy-efficient transmissions that satisfy the latency constraints for V2V links while reducing the interference of the V2V network to the V2I network. We evaluate the performance of the proposed scheme in terms of the sum rate of the V2X network, the average power consumption of V2V links, and the average outage probability of V2V links using a case study in Manhattan with nine blocks of 3GPP TR 36.885. The simulation results show that the proposed scheme greatly reduces the transmit power of V2V links when compared to the conventional reinforcement learning-based resource allocation scheme without sacrificing the sum rate of the V2X network or the outage probability of V2V links.