Using Deep Reinforcement Learning with Hierarchical Risk Parity for Portfolio Optimization
Open Access
- 29 December 2022
- journal article
- research article
- Published by MDPI AG in International Journal of Financial Studies
- Vol. 11 (1) , 10
Abstract
We devise a hierarchical decision-making architecture for portfolio optimization on multiple markets. At the highest level a Deep Reinforcement Learning (DRL) agent selects among a number of discrete actions, representing low-level agents. For the low-level agents, we use a set of Hierarchical Risk Parity (HRP) and Hierarchical Equal Risk Contribution (HERC) models with different hyperparameters, which all run in parallel, off-market (in a simulation). The information on which the DRL agent decides which of the low-level agents should act next is constituted by the stacking of the recent performances of all agents. Thus, the modelling resembles a statefull, non-stationary, multi-arm bandit, where the performance of the individual arms changes with time and is assumed to be dependent on the recent history. We perform experiments on the cryptocurrency market (117 assets), on the stock market (46 assets) and on the foreign exchange market (28 pairs) showing the excellent robustness and performance of the overall system. Moreover, we eliminate the need for retraining and are able to deal with large testing sets successfully.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EP/L016796/1)
This publication has 24 references indexed in Scilit:
- Bayesian estimation of the global minimum variance portfolioEuropean Journal of Operational Research, 2017
- Building Diversified Portfolios that Outperform Out of SampleThe Journal of Portfolio Management, 2016
- Deep Direct Reinforcement Learning for Financial Signal Representation and TradingIEEE Transactions on Neural Networks and Learning Systems, 2016
- Human-level control through deep reinforcement learningNature, 2015
- 60 Years of portfolio optimization: Practical challenges and current trendsEuropean Journal of Operational Research, 2014
- Minimum-Variance Portfolio CompositionThe Journal of Portfolio Management, 2011
- The Properties of Equally Weighted Risk Contribution PortfoliosThe Journal of Portfolio Management, 2010
- Toward Maximum DiversificationThe Journal of Portfolio Management, 2008
- Improved estimation of the covariance matrix of stock returns with an application to portfolio selectionJournal of Empirical Finance, 2003
- Global Portfolio OptimizationCFA Magazine, 1992