从噪声数据学习偏微分方程的复合神经网络

doi:10.19596/j.cnki.1001-246x.8364

摘要/Abstract

摘要：

提出一种名为NN-PDE(neural network-partial differential equations)的复合神经网络方法, 用于噪声数据预处理和学习偏微分方程。NN-PDE用一套神经网络负责数据预处理, 另一套网络耦合备选的方程信息, 进而学习潜在的控制方程。两套网络复合为一套网络, 可更加高效地处理噪声数据, 有效减小噪声的影响。使用NN-PDE学习多种物理方程(如Burgers方程、Korteweg-de Vries方程、Kuramoto-Sivashinsky方程和Navier-Stokes方程)的噪声数据, 均可获得准确的控制方程。

关键词: 噪声数据, 复合神经网络, 偏微分方程

Abstract:

A compound neural network method, NN-PDE(neural network-partial differential equations), is proposed for data preprocessing and learning partial differential equation. NN-PDE uses one sub network to preprocess noisy data, and another one to couple information of the alternative equations to learn the underlying governing equation. These two sub networks are merged into one compound network so that it can process noisy data more efficiently and effectively to reduce the influence of noise. Noisy data generated from various physical equations (such as Burgers equation, Korteweg-de Vries(KdV) equation, Kuramoto-Sivashinsky(KS) equation and Navier-Stokes(NS) equation) are studied with NN-PDE, and accurate governing equations are obtained.

Key words: noisy data, compound neural network, partial differential equations

潘剑, 郭照立, 陈松泽. 从噪声数据学习偏微分方程的复合神经网络[J]. 计算物理, 2022, 39(2): 223-232.

Jian PAN, Zhaoli GUO, Songze CHEN. A Compound Neural Network for Learning Partial Differential Equations from Noisy Data[J]. Chinese Journal of Computational Physics, 2022, 39(2): 223-232.

图/表 10

图1 全连接神经网络结构

Fig.1 Structure of a fully connected neural network

图2 复合神经网络结构(蓝色部分表示逼近方程解的网络，黄色部分表示构造的候选项，DATA部分表示数据的残差，PDE部分表示构造的控制方程残差。)

Fig.2 Structure of a compound neural network (The blue part represents the network that approximates solution of the equation. The yellow part represents the constructed candidate library. The DATA part represents residual of the data. The PDE part represents residual of the constructed governing equation.)

图3 (a) Burgers方程解u的云图；(b) 图中一部分代表从数据中随机选择的3000组 (背景为解u的云图，黑点是选择的数据。)

Fig.3 (a) Solution of Burgers equation in the form of heat map and (b) a portion of the map indicating 3000 data selected randomly from the dataset (The background represents the solution u in the dataset by heat map, and the black dots are selected data.)

表1 Burgers方程ut=- uux+0.1uxx在不同噪声中NN-PDE和STRidge的性能

Table 1 Performance of NN-PDE and STRidge in different noises for Burgers equation ut=- uux+0.1uxx

噪声数据量	NN-PDE	STRidge
无	u_t=- 0.9988 uu_x+0.0999 u_xx	u_t=- 1.001 uu_x+0.102 u_xx
1%	u_t=- 0.9965 uu_x+0.0997 u_xx	u_t=- 0.960 uu_x+0.102 u_xx
5%	u_t=- 0.9992 uu_x+0.1003 u_xx	u_t=- 0.824 uu_x+0.117 u_xx - 0.06 u²u_xx
10%	u_t=- 0.9789 uu_x+0.0998 u_xx	无法得到简约的方程

图4 (a) KdV方程解u的云图；(b) 图中一部分代表从数据中随机选择的25 000组 (背景代表解u的云图，黑点是选择的数据。)

Fig.4 (a) Solution of KdV equation in the form of heat map and (b) a portion of the map indicating 25 000 data selected randomly from the dataset (The background represents solution u in the dataset by heat map, and the black dots are selected data.)

表2 KdV方程ut=- uux - uxxx在不同噪声中NN-PDE和DL-PDE的性能

Table 2 Performance of NN-PDE and STRidge in different noises for KdV equation ut=- uux - uxxx

噪声数据量	NN-PDE	DL-PDE
无	u_t=- 0.9995 uu_x - 0.9995 u_xxx	u_t=- 0.993 uu_x - 0.996 u_xxx
1%	u_t=- 0.9986 uu_x - 0.9991 u_xxx	u_t=- 0.986 uu_x - 0.985 u_xxx
5%	u_t=- 0.9995 uu_x - 0.9973 u_xxx	u_t=- 0.968 uu_x - 0.974 u_xxx
10%	u_t=- 1.0018 uu_x - 1.0023 u_xxx	u_t=- 0.940 uu_x - 0.936 u_xxx

图5 (a) KS方程解u的云图；(b)图的一部分代表从数据中随机选择的25 000组 (背景代表解u的云图，黑点是选择的数据。)

Fig.5 (a) Solution of KS equation in the form of heat map and (b) a portion of the map indicating 25 000 data selected randomly from the dataset (The background represents the solution u in the dataset by heat map, and the black dots are selected data.)

表3 KS方程ut=- uux - uxx - uxxxx在不同噪声中NN-PDE的性能

Table 3 Performance of NN-PDE and STRidge in different noises for KS equation ut=- uux - uxx - uxxxx

噪声数据量	控制方程
无	u_t=- 0.9969 uu_x - 0.9966 u_xx - 0.9972 u_xxxx
1%	u_t=- 0.9982 uu_x - 0.9954 u_xx - 0.9964 u_xxxx
5%	u_t=- 0.9962 uu_x - 0.9946 u_xx - 0.9955 u_xxxx
10%	u_t=- 0.9780 uu_x - 1.0020 u_xx - 0.9964 u_xxxx

图6 (a) NS方程解w的云图；(b)图中一部分代表从数据中随机选择的50 000组 (背景代表解w的云图，黑点是选择的数据。)

Fig.6 (a) Solution of NS equation in the form of heat map and (b) a portion of the map indicating 50 000 data selected randomly from the dataset (The background represents the solution w in the dataset by heat map, and the black dots are selected data.)

表4 NS方程wt=- uwx - vwy+0.01(wxx+wyy) 在不同噪声中NN-PDE的性能

Table 4 Performance of NN-PDE and STRidge in different noises for NS equation wt=- uwx - vwy+0.01(wxx+wyy)

噪声数据量	控制方程
无	w_t=- 0.9995 uw_x- 0.9986 vw_y+0.0100 w_xx+0.0100 w_yy
1%	w_t=- 0.9993 uw_x- 0.9982 vw_y+0.0100 w_xx+0.0100 w_yy
5%	w_t=- 0.9988 uw_x- 0.9919 vw_y+0.0100 w_xx+0.0100 w_yy
10%	w_t=- 0.9936 uw_x- 0.9729 vw_y+0.0095 w_xx+0.0108 w_yy

参考文献 23

1	BRUNTON S L , PROCTOR J L , KUTZ J N . Discovering governing equations from data by sparse identification of nonlinear dynamical systems[J]. Proceedings of the National Academy of Sciences, 2016, 113 (15): 3932- 3937. DOI
2	SCHAEFFER H . Learning partial differential equations via data discovery and sparse optimization[J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2017, 473 (2197): 20160446. DOI
3	RUDY S H , BRUNTON S L , PROCTOR J L , et al. Data-driven discovery of partial differential equations[J]. Science Advances, 2017, 3 (4): e1602614. DOI
4	MANGAN N M , KUTZ J N , BRUNTON S L , et al. Model selection for dynamical systems via sparse regression and information criteria[J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2017, 473 (2204): 20170009. DOI
5	BONINSEGNA L , NUSKE F , CLEMENTI C . Sparse learning of stochastic dynamic equations[J]. The Journal of Chemical Physics, 2018, 148 (24): 241723. DOI
6	QUADE M , ABEL M , KUTZ J N , et al. Sparse identification of nonlinear dynamics for rapid model recovery[J]. Chaos: An Interdisciplinary Journal of Nonlinear Science, 2018, 28 (6): 063116. DOI
7	RUDY S , ALLA A , BRUNTON S L , et al. Data-driven identification of parametric partial differential equations[J]. SIAM Journal on Applied Dynamical Systems, 2019, 18 (2): 643- 660. DOI
8	CHAMPION K , LUSCH B , KUTZ J N , et al. Data-driven discovery of coordinates and governing equations[J]. Proceedings of the National Academy of Sciences of the United States of America, 2019, 116 (45): 22445- 22451. DOI
9	BERG J , NYSTROM K . Data-driven discovery of PDEs in complex datasets[J]. Journal of Computational Physics, 2019, 384, 239- 252. DOI
10	CHANG H , ZHANG D . Machine learning subsurface flow equations from data[J]. Computational Geosciences, 2019, 23 (5): 895- 910. DOI
11	CHANG H , ZHANG D . Identification of physical processes via combined data-driven and data-assimilation methods[J]. Journal of Computational Physics, 2019, 393, 337- 350. DOI
12	CHARTRAND R . Numerical differentiation of noisy, nonsmooth data[J]. ISRN Applied Mathematics, 2011, 2011, 1023- 1033.
13	BRUNO O , HOCH D . Numerical differentiation of approximated functions with limited order-of-accuracy deterioration[J]. SIAM Journal on Numerical Analysis, 2012, 50 (3): 1581- 1603. DOI
14	KNOWLES I, RENKA R J. Methods for numerical differentiation of noisy data[J]. Electronic Journal of Differential Equations, Conference 21(2014): 235-246.
15	SCHAEFFER H , MCCALLA S G . Sparse model selection via integral terms[J]. Physical Review E, 2017, 96 (2): 023302. DOI
16	RAISSI M . Machine learning of linear differential equations using Gaussian processes[J]. Journal of Computational Physics, 2017, 348, 683- 693. DOI
17	RAISSI M , KARNIADAKIS G E . Hidden physics models: Machine learning of nonlinear partial differential equations[J]. Journal of Computational Physics, 2018, 357, 125- 141. DOI
18	RAISSI M , PERDIKARIS P , KARNIADAKIS G E . Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational Physics, 2019, 378, 686- 707. DOI
19	RAISSI M . Deep hidden physics models: Deep learning of nonlinear partial differential equations[J]. Journal of Machine Learning Research, 2018, 19 (25): 1- 24.
20	LONG Z , LU Y , MA X , et al. PDE-NET: Learning PDEs from data[J]. PMLR, 2018, 80, 3208- 3216.
21	XU H . DL-PDE: Deep-learning based data-driven discovery of partial differential equations from discrete and noisy data[J]. Communications in Computational Physics, 2021, 29 (3): 698- 728. DOI
22	CYBENKO G . Approximation by superpositions of a sigmoidal function[J]. Mathematics of Control Signals & Systems, 1989, 2 (4): 303- 314.
23	GLOROT X , BENGIO Y . Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research, 2010, 9, 249- 256.

[1]	胡军, 刘全, 倪国喜. 时变偏微分方程的贝叶斯稀疏识别方法[J]. 计算物理, 2021, 38(1): 25-34.
[2]	孔令海, 孔令波, 许海波, 贾清刚. 含Laplace-Gauss型混合噪声图像二阶正则化重建方法[J]. 计算物理, 2019, 36(3): 280-290.
[3]	梅树立, 张森文, 陆启韶. 基于同伦技术的Burgers方程的小波精细积分算法[J]. 计算物理, 2007, 24(1): 54-58.
[4]	梅树立, 陆启韶, 张森文. 求解非线性偏微分方程的自适应小波精细积分法[J]. 计算物理, 2004, 21(6): 523-530.
[5]	李晓峰, 朱本仁. Radon变换在渗流力学反问题中的应用[J]. 计算物理, 2001, 18(2): 115-118.
[6]	史金松, 庄卫华, 章卫军. Kaczmarz迭代与流场计算[J]. 计算物理, 1991, 8(3): 279-286.