Active Learning Algorithm Using Neural Operator Networks and Bayesian Neural Networks: Learning Macroscale Models for Collective Behavior from Microscale Data

doi:10.19596/j.cnki.1001-246x.8979

Abstract

Abstract:

With the development of artificial intelligence and scientific computing, deep learning plays a significant role in mathematical modeling. In this work we develop an active learning algorithm that uses microscopic data to establish a macroscopic model for collective behavior. Specifically, we take the Cucker-Smale model in this work and develop the corresponding active learning algorithm that integrates neural operator networks and Bayesian neural networks by utilizing microscopic particle data and partial physics. This algorithm is used to efficiently establishes the corresponding macroscopic Euler model through microscopic data. Finally, the effectiveness of the active learning algorithm is validated through one-dimensional and two-dimensional numerical simulations.

Key words: mathematical modeling, nonlocal Euler equations, Bayesian neural networks, active learning algorithm, Cucker-Smale model

Zhengya GAO, Zhiping MAO. Active Learning Algorithm Using Neural Operator Networks and Bayesian Neural Networks: Learning Macroscale Models for Collective Behavior from Microscale Data[J]. Chinese Journal of Computational Physics, 2024, 41(6): 783-796.

Figures/Tables 8

Fig.1 Schematic of DeepONet

Fig.2 Prediction of DeepONet for one-dimensional Euler equation (a)loss of train and test; (b) p rediction of u with α=1; (c)exact and predict solution of u with α=1 at t=0.5, 1.5, 2.5

Fig.3 Prediction of the DeepONet for two-dimensional Euler equation (a)train and test loss of u and v; (b) test loss of u and v with α=1 and t=0.5, 1.2

Fig.4 Schematic of Bayesian neural networks

Table 1 Inferred value of α for the one-dimensional case

$\hat{\alpha}$	α	F(α)
0.5	0.481 5	3.04×10^-2
1.2	1.210 4	5.05×10^-2

Fig.5 Learning process of $\hat{\alpha}$=0.5, 1.2

Table 2 Inferred value of α for the two-dimensional case

$\hat{\alpha}$	α	F(α)
0.5	0.550 2	1.34×10^-1
1.2	1.253 8	8.79×10^-2

Fig.6 Learning process of $\hat{\alpha}$ =0.5, 1.2

References 29

1	鄂维南. AI助力打造科学研究新范式[J]. 中国科学院院刊, 2024, 39 (1): 10- 16.
2	BRUNTON S L , PROCTOR J L , KUTZ J N . Discovering governing equations from data by sparse identification of nonlinear dynamical systems[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016, 113 (15): 3932- 3937.
3	CARLEO G , TROYER M . Solving the quantum many-body problem with artificial neural networks[J]. Science, 2017, 355 (6325): 602- 606. DOI
4	RUDY S H , BRUNTON S L , PROCTOR J L , et al. Data-driven discovery of partial differential equations[J]. Science Advances, 2017, 3 (4): e1602614. DOI
5	E Weinan, YU BING. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems[DB/OL]. arXiv, 2017(2017-09-30). https://arxiv.org/abs/1710.00211.
6	HAN Jiequn , JENTZEN A , E Weinan . Solving high-dimensional partial differential equations using deep learning[J]. Proceedings of the National Academy of Sciences of the United States of America, 2018, 115 (34): 8505- 8510.
7	RAISSI M , PERDIKARIS P , KARNIADAKIS G E . Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational Physics, 2019, 378, 686- 707. DOI
8	SIRIGNANO J , SPILIOPOULOS K . DGM: A deep learning algorithm for solving partial differential equations[J]. Journal of Computational Physics, 2018, 375, 1339- 1364. DOI
9	ZANG Yaohua , BAO Gang , YE Xiaojing , et al. Weak adversarial networks for high-dimensional partial differential equations[J]. Journal of Computational Physics, 2020, 411, 109409. DOI
10	BAO G , YE X , ZANG Y , et al. Numerical solution of inverse problems by weak adversarial net-works[J]. Inverse Problems, 2020, 36 (11): 115003. DOI
11	CUCKER F , SMALE S . Emergent behavior in flocks[J]. IEEE Transactions on Automatic Control, 2007, 52 (5): 852- 862. DOI
12	何大韧, 刘宗华, 汪秉宏. 复杂系统与复杂网络[M]. 北京: 高等教育出版社, 2009.
13	DEMIDOVA L , GORCHAKOV A . Research and study of the hybrid algorithms based on the collective behavior of fish schools and classical optimization methods[J]. Algorithms, 2020, 13 (4): 85. DOI
14	MANN R P , GARNETT R . The entropic basis of collective behaviour[J]. Journal of the Royal Society, Interface, 2015, 12 (106): 20150037. DOI
15	VICSEK T , CZIRÓK A , BEN-JACOB E , et al. Novel type of phase transition in a system of self-driven particles[J]. Physical Review Letters, 1995, 75 (6): 1226- 1229. DOI
16	COUZIN I D , KRAUSE J , JAMES R , et al. Collective memory and spatial sorting in animal groups[J]. Journal of Theoretical Biology, 2002, 218 (1): 1- 11. DOI
17	HA S Y , LIU Jianguo . A simple proof of the Cucker-Smale flocking dynamics and mean-field limit[J]. Communications in Mathematical Sciences, 2009, 7 (2): 297- 325. DOI
18	KIM J H , PARK J H . Complete characterization of flocking versus nonflocking of Cucker–Smale model with nonlinear velocity couplings[J]. Chaos, Solitons & Fractals, 2020, 134, 109714.
19	MOTSCH S , TADMOR E . A new model for self-organized dynamics and its flocking behavior[J]. Journal of Statistical Physics, 2011, 144 (5): 923. DOI
20	HASKOVEC J . Flocking dynamics and mean-field limit in the Cucker–Smale-type model with topological interactions[J]. Physica D. Nonlinear Phenomena, 2013, 261, 42- 51. DOI
21	SHEN J . Cucker-Smale flocking under hierarchical leadership[J]. SIAM Journal on Applied Mathematics, 2008, 68 (3): 694- 719. DOI
22	LI Zhuchun , XUE Xiaoping . Cucker–Smale flocking under rooted leadership with fixed and switching topologies[J]. SIAM Journal on Applied Mathematics, 2010, 70 (8): 3156- 3174. DOI
23	GIARDINA I . Collective behavior in animal groups: theoretical models and empirical studies[J]. HFSP Journal, 2008, 2 (4): 205- 219. DOI
24	MAO Zhiping , LI Zhen , KARNIADAKIS G E . Nonlocal flocking dynamics: learning the fractional order of PDEs from particle simulations[J]. Communications on Applied Mathematics and Computation, 2019, 1 (4): 597- 619. DOI
25	PESZEK J . Discrete Cucker-Smale's flocking model with a weakly singular weight[J]. SIAM Journal on Mathematical Analysis, 2015, 47 (5): 3671- 3686. DOI
26	FIGALLI A , KANG M J . A rigorous derivation from the kinetic Cucker-Smale model to thepressureless Euler system with nonlocal alignment[J]. Anal. PDE, 2019, 12 (3): 843- 866. DOI
27	SHVYDKOY R , TADMOR E . Eulerian dynamics with a commutator forcing[J]. Transactions of Mathematics and Its Applications, 2017, 1 (1): 1- 26.
28	LU Lu , JIN Pengzhan , PANG Guofei , et al. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators[J]. Nature Machine Intelligence, 2021, 3 (3): 218- 229. DOI
29	YANG Liu , MENG Xuhui , KARNIADAKIS G E . B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data[J]. Journal of Computational Physics, 2021, 425, 109913. DOI