A multi-agent deep reinforcement learning method for co2 flooding rates optimization
Open Access
- 15 July 2022
- journal article
- research article
- Published by SAGE Publications in Energy Exploration & Exploitation
- Vol. 41 (1) , 224-245
Abstract
The CO2 flooding with superior displacement efficiency and high injectivity is an efficient enhanced oil recovery method. However, due to the unfavorable sweep efficiency particularly for strong heterogeneous reservoirs and immiscible flooding, the oil recovery on site is not all favorable. Multi-well rates optimization is one of common measures improving sweep efficiency with easy implement and low cost There are many rates optimization methods have been proposed by now. In this research, we first introduced the multi-agent deep deterministic policy gradient (MADDPG) algorithm to the multi-well rates optimization of CO2 flooding, and the new rates optimization method was built. The MADDPG adopts the centralized training and decentralized execution algorithm framework, and overcomes the defect that the single-agent reinforcement learning cannot deal the multi-well rates optimization well and also avoids the dimensional disaster problems. We treated each well as an agent, and each agent has its own reward, state and action. We chose the net present value (NPV) as the reward, the injection-production rate change range as the action element, and the production time, the bottom hole pressure, the oil production rate, and the gas-oil ratio as the state elements. The simulation results show that the optimal case obviously improves the NPV compared with the base case, and the simulation case with strong heterogeneity and immiscible flooding can also converge to the optimal target, which prove the effectiveness and robustness of the rates optimization method respectively. This research provides recommendations that improving the oil recovery by increasing the sweep efficiency to increase the income, and reducing the invalid CO2 injection to decrease the cost can achieve the optimal NPV. Reservoir heterogeneity seriously impairs the rates optimization performance, and rates optimization makes little difference to extreme strong interlayer heterogeneity for serious interlayer influence.Keywords
Funding Information
- National Science and Technology Major Projects (2017ZX05009004-005)
- National Natural Science Foundation of China (U1762210)
This publication has 30 references indexed in Scilit:
- Reinforcement learning and optimal adaptive control: An overview and implementation examplesAnnual Reviews in Control, 2012
- Adjoint-based optimization of multi-phase flow through porous media – A reviewComputers & Fluids, 2011
- Optimization of Production Performance in a CO2 Flooding Reservoir Under UncertaintyJournal of Canadian Petroleum Technology, 2010
- First-Contact-Miscible and Multicontact-Miscible Gas Injection within a Channeling Heterogeneity SystemEnergy & Fuels, 2010
- Policy gradient methodsScholarpedia, 2010
- Enhanced Oil Recovery by CO2 Flooding in Homogeneous and Heterogeneous 2D MicromodelsJournal of Canadian Petroleum Technology, 2009
- Evolution of the Carbon Dioxide Flooding ProcessesJournal of Petroleum Technology, 1987
- The Effect of Phase Behavior on CO2-Flood Displacement EfficiencyJournal of Petroleum Technology, 1981
- Mechanisms of Oil Displacement By Carbon DioxideJournal of Petroleum Technology, 1974
- Prediction of Recovery in Unstable Miscible FloodingSociety of Petroleum Engineers Journal, 1972