Quantum architecture search via truly proximal policy optimization
Authors:
Open Access
- 29 March 2023
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 13 (1) , 1-18
Abstract
Quantum Architecture Search (QAS) is a process of voluntarily designing quantum circuit architectures using intelligent algorithms. Recently, Kuo et al. (Quantum architecture search via deep reinforcement learning. arXiv preprint arXiv:2104.07715, 2021) proposed a deep reinforcement learning-based QAS (QAS-PPO) method, which used the Proximal Policy Optimization (PPO) algorithm to automatically generate the quantum circuit without any expert knowledge in physics. However, QAS-PPO can neither strictly limit the probability ratio between old and new policies nor enforce well-defined trust domain constraints, resulting in poor performance. In this paper, we present a new deep reinforcement learning-based QAS method, called Trust Region-based PPO with Rollback for QAS (QAS-TR-PPO-RB), to automatically build the quantum gates sequence from the density matrix only. Specifically, inspired by the research work of Wang, we employ an improved clipping function to implement the rollback behavior to limit the probability ratio between the new strategy and the old strategy. In addition, we use the triggering condition of the clipping based on the trust domain to optimize the policy by restricting the policy within the trust domain, which leads to guaranteed monotone improvement. Experiments on several multi-qubit circuits demonstrate that our presented method achieves better policy performance and lower algorithm running time than the original deep reinforcement learning-based QAS method.Keywords
This publication has 63 references indexed in Scilit:
- Learning in quantum control: High-dimensional global optimization for noisy quantum dynamicsNeurocomputing, 2017
- Mastering the game of Go without human knowledgeNature, 2017
- Quantum computational supremacyNature, 2017
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Human-level control through deep reinforcement learningNature, 2015
- A variational eigenvalue solver on a photonic quantum processorNature Communications, 2014
- Quantum Computation and Quantum InformationAmerican Journal of Physics, 2002
- Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum ComputerSIAM Review, 1999
- Quantum Mechanics Helps in Searching for a Needle in a HaystackPhysical Review Letters, 1997
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992