An Efficient Approach for Automatic Well-testing Interpretation Based on Surrogate Model and Deep Reinforcement Learning

doi:10.19596/j.cnki.1001-246x.8533

Abstract

Abstract:

A deep reinforcement learning (DRL) based approach is proposed for automatic interpretation of well-testing curves. Based on a deep deterministic policy gradient (DDPG) algorithm, the proposed DRL approach is successfully applied to automatic matching of four different types of well-testing curves. To improve training efficiency, a surrogate well-testing model based on LSTM neural network was established. With episodic training, through interaction with the surrogate model the agent converged finally to an optimal curve matching policy on different well-testing models. It shows that the average relative error of the curve parameter interpretation is 5.51%. Additionally, the proposed DRL approach has a high calculation speed, and the average computing time is 0.27 seconds. In case study applications, the proposed method achieved an average relative error of 4.32% in parameter interpretation, which shows reliability of the method.

Key words: well testing, parametric inversion, deep reinforcement learning, automatic matching

Peng DONG, Xinwei LIAO. An Efficient Approach for Automatic Well-testing Interpretation Based on Surrogate Model and Deep Reinforcement Learning[J]. Chinese Journal of Computational Physics, 2023, 40(1): 67-80.

Figures/Tables 20

References 35

1	WEI X X , FENG Q H , ZHANG X M , et al. Distribution of remaining oil in water flooding at pore scale: Volume of fluid method[J]. Chinese Journal of Computational Physics, 2021, 38 (5): 573- 584.
2	ZHANG J , CHENG S Q , ZENG Y , et al. Applicability exploration of superposition principle for pressure transient analysis in polymer-flooding system[J]. Chinese Journal of Computational Physics, 2021, 38 (3): 324- 332.
3	严涛. 试井多解性问题研究[J]. 海洋石油, 2008, (3): 46- 50.
4	KHADIVI K , HASSANZADEH H . Role of integration of core data in elimination of nonunique solutions in well testing of heterogeneous formations[J]. Journal of Porous Media, 2021, 24 (1): 25- 40. DOI
5	文必龙, 李艳春. 基于大数据的试井解释参数分析[J]. 计算机应用与软件, 2019, 36 (9): 64- 69.
6	张晓东. 试井曲线计算机自动拟合算法及应用[J]. 西南石油大学学报(自然科学版), 2008, (4): 135- 137.
7	王一, 陈殿远, 韩鑫, 等. 基于信赖域算法的试井曲线自动拟合研究[J]. 重庆科技学院学报(自然科学版), 2019, 21 (4): 47- 50. DOI
8	DASTAN A , HORNE R . Robust well-test interpretation by using nonlinear regression with parameter and data transformations[J]. SPE Journal, 2011, 16 (3): 698- 712. DOI
9	吴明录, 姚军, 王子胜, 等. 利用双种群遗传算法进行数值试井自动拟合[J]. 油气地质与采收率, 2007, (2): 81- 83. DOI
10	王培玺, 张静. 基于分群式粒子群算法的压裂水平井试井曲线自动拟合[J]. 中国石油大学学报(自然科学版), 2012, 36 (2): 136- 140. DOI
11	AWOTUNDE A A . Estimation of well test parameters using global optimization techniques[J]. Journal of Petroleum Science and Engineering, 2015, 125, 269- 277. DOI
12	GOMEZ S, CAMACHO R, VASQUEZ M, et al. Well test characterization of naturally fractured vuggy reservoirs, with a global optimization method[C]. Offshore Technology Conference, 2014.
13	胡修凤. 基于BP神经网络的沁中南煤层气井解吸压力预测及应用[D]. 焦作: 河南理工大学, 2017.
14	ALMARAGHI A M, EL-BANBI A H. Automatic reservoir model identification using artificial neural networks in pressure transient analysis[C]. Society of Petroleum Engineers, 2015.
15	MOOSAVI S R , VAFERI B , WOOD D A . Auto-detection interpretation model for horizontal oil wells using pressure transient responses[J]. Advances in Geo-Energy Research, 2020, 4 (3): 305- 316. DOI
16	VAFERI B , ESLAMLOUEYAN R , AYATOLLAHI S . Automatic recognition of oil reservoir models from well testing data by using multi-layer perceptron networks[J]. Journal of Petroleum Science and Engineering, 2011, 77 (3-4): 254- 262. DOI
17	糜利栋, 顾少华, 薛亮, 等. 基于数据驱动技术的智能试井解释方法——以有水气藏产水气井为例[J]. 天然气工业, 2021, 41 (2): 119- 124.
18	XUE L , GU S , MI L , et al. An automated data-driven pressure transient analysis of water-drive gas reservoir through the coupled machine learning and ensemble Kalman filter method[J]. Journal of Petroleum Science and Engineering, 2022, 208, 109492. DOI
19	李道伦, 刘旭亮, 查文舒, 等. 基于卷积神经网络的径向复合油藏自动试井解释方法[J]. 石油勘探与开发, 2020, 47 (3): 583- 591.
20	LIU X , LI D , YANG J , et al. Automatic well test interpretation based on convolutional neural network for infinite reservoir[J]. Journal of Petroleum Science and Engineering, 2020, 195, 107618. DOI
21	周子琪. 基于深度学习的试井和测井资料解释方法研究[D]. 合肥: 合肥工业大学, 2021.
22	DONG P , CHEN Z , LIAO X , et al. Application of deep learning on well-test interpretation for identifying pressure behavior and characterizing reservoirs[J]. Journal of Petroleum Science and Engineering, 2021, 109264.
23	ZHANG Q , ZHU S . Visual interpretability for deep learning: A survey[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19 (1): 27- 39.
24	SUTTON R S , BARTO A G . Reinforcement learning: An introduction[M]. Cambridge: MIT Press, 2018.
25	LILLICRAP T P , HUNT J J , PRITZEL A , et al. Continuous control with deep reinforcement learning[J]. arXiv Preprint arXiv: 1509.02971, 2015,
26	HOURFAR F , BIDGOLY H J , MOSHIRI B , et al. A reinforcement learning approach for waterflooding optimization in petroleum reservoirs[J]. Engineering Applications of Artificial Intelligence, 2019, 77, 98- 116.
27	HE J , TANG M , HU C , et al. Deep reinforcement learning for generalizable field development optimization[J]. SPE Journal, 2021, 1- 20.
28	GUEVARA J L , PATEL R . Optimization of steam injection in SAGD using reinforcement learning[J]. Journal of Petroleum Science and Engineering, 2021, 108735.
29	ZHANG Y Y , LI W W , CHEN H J . Reduced-order model based on cluster analysis in numerical simulation of low permeability reservoir[J]. Chinese Journal of Computational Physics, 2015, 32 (5): 603- 609.
30	HUANG C , TIAN L , WANG H L , et al. A single well production forecasting model of reservoir based on conditional generative adversarial net[J]. Chinese Journal of Computational Physics, 2022, 39 (4): 465- 478.
31	TANG M , LIU Y , DURLOFSKY L J . A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems[J]. Journal of Computational Physics, 2020, 413, 109456.
32	XIAO C , ZHANG S C , MA X F , et al. Model-reduced autoregressive neural network for parameter inversion[J]. Chinese Journal of Computational Physics, 2022, 39 (5): 564- 578.
33	ZHONG Z , SUN A Y , REN B , et al. A deep-learning-based approach for reservoir production forecast under uncertainty[J]. SPE Journal, 2021, 26 (3): 1314- 1340.
34	KINGMA D P , BA J . Adam: A method for stochastic optimization[J]. arXiv Preprint arXiv: 1412.6980, 2014,
35	MNIH V , KAVUKCUOGLU K , SILVER D , et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518 (7540): 529- 533.

参数	最小值	最大值	分布	模型1	模型2	模型3	模型4
C_D	20	2 000	对数均匀分布	√	√	√	√
S	0.01	1	均匀分布	√	√	√	√
L_xf/m	30	300	对数均匀分布	√	√	√	√
C_fD	400	25 000	对数均匀分布	√	√	√	√
n_f	5	25	均匀分布		√	√	√
L_w/m	200	2 000	均匀分布		√	√	√
ω	0.01	0.2	均匀分布			√
λ	10^-8	10^-4	对数均匀分布			√
R_i/m	200	4 000	均匀分布				√
M	2	10	均匀分布				√

参数	最小值	最大值	分布	模型1	模型2	模型3	模型4
C_D	20	2 000	对数均匀分布	√	√	√	√
S	0.01	1	均匀分布	√	√	√	√
L_xf/m	30	300	对数均匀分布	√	√	√	√
C_fD	400	25 000	对数均匀分布	√	√	√	√
n_f	5	25	均匀分布		√	√	√
L_w/m	200	2 000	均匀分布		√	√	√
ω	0.01	0.2	均匀分布			√
λ	10^-8	10^-4	对数均匀分布			√
R_i/m	200	4 000	均匀分布				√
M	2	10	均匀分布				√

试井模型	物理模型计算时间/s	代理模型
试井模型	物理模型计算时间/s	训练时间/min	预测时间/s
垂直裂缝	4.23	7.1	0.018
水平井	7.71	7.5	0.023
双重介质水平井	13.63	8.2	0.021
径向复合水平井	12.87	7.7	0.027

试井模型	物理模型计算时间/s	代理模型
试井模型	物理模型计算时间/s	训练时间/min	预测时间/s
垂直裂缝	4.23	7.1	0.018
水平井	7.71	7.5	0.023
双重介质水平井	13.63	8.2	0.021
径向复合水平井	12.87	7.7	0.027

试井模型	变量	训练时间/min
试井模型	变量	代理模型训练	智能体训练
垂直裂缝	C_D, S, L_xf, C_fD	6.3	67
水平井	C_D, S, L_xf, C_fD	7.5	70
双重介质水平井	L_xf, C_fd, ω, λ	7.1	68
径向复合水平井	L_xf, C_fd, R_i, M	7.6	72