Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

Authors:
Publisher website
Google Scholar
Abstract
In this paper, an IRS-aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-threading version in terms of transmit beamforming and phase-shift design.
All Related Versions
Funding Information
  • National Key Research and Development Program of China (2020YFB1708800)
  • Beijing Natural Science Foundation (L212004)
  • National Natural Science Foundation of China (61822104, 61771044)
  • State Key Laboratory of Advanced Metallurgy (KF2004)
  • Fundamental Research Funds for the Central Universities (FRFTP-19-002C1, RC1631)
  • Beijing Top Discipline for Artificial Intelligent Science and Engineering, University of Science and Technology Beijing

This publication has 36 references indexed in Scilit: