Causal Physics-Informed Neural Networks for Singular and Singularly Perturbed Boundary Value Problems in Chemical Systems
DOI:
https://doi.org/10.63623/n5jjnf09Keywords:
Scientific machine learning, Causality, PINNs, Chemical differential equations, SBVPs and SPBVPsAbstract
Singular boundary value problems (SBVPs) and singularly perturbed boundary value problems (SPBVPs) are commonly arise in the modeling of critical chemical processes, including isothermal gas spheres, electroactive polymer films, thermal explosions, and chemical reactor theory. Due to their extensive applications, these problems require the creation of effective numerical methods for precise solutions. Traditional methods often struggle with stiffness and multiscale behavior, while analytical solutions are limited to specific cases. In this work, we use a data-driven approach using Physics-Informed Neural Networks (PINNs) to solve SBVP and SPBVPs. By initializing the perturbation parameter as a trainable variable, it is optimized alongside net-work weights and biases during training. Numerical simulations demonstrate that causality-enhanced PINNs significantly improve the solution accuracy for singular chemical boundary value problems.
References
[1]Mehrpouya MA. An efficient pseudospectral method for numerical solution of nonlinear singular initial and boundary value problems arising in astrophysics. Mathematical Methods in the Applied Sciences, 2016, 39(12), 3204-3214. DOI: 10.1002/mma.3763
[2]Ali MR, Hadhoud AR, Ma WX. Evolutionary numeri-cal approach for solving nonlinear singular periodic boundary value problems. Journal of Intelligent & Fuzzy Systems, 2020, 39(5), 7723-7731. DOI: 10.3233/JIFS-201045
[3]Adam JA, Maggelakis SA. Mathematical models of tumor growth. IV. Effects of a necrotic core. Mathematical Biosciences, 1989, 97(1), 121-136. DOI: 10.1016/0025-5564(89)90045-X
[4]Roul P. A fourth-order non-uniform mesh optimal B-spline collocation method for solving a strongly nonlinear singular boundary value problem de-scribing electrohydrodynamic flow of a fluid. Applied Numerical Mathematics, 2020, 153, 558-574. DOI: 10.1016/j.apnum.2020.03.018
[5]Kumar M, Singh N. Modified adomian decomposition method and computer implementation for solving singular boundary value problems arising in various physical problems. Computers & Chemical Engineering, 2010, 34(11), 1750-1760. DOI: 10.1016/j.compchemeng.2010.02.035
[6]Khuri SA, Sayfy A. A novel approach for the solution of a class of singular boundary value problems arising in physiology. Mathematical and Computer Modelling, 2010, 52(3-4), 626-636. DOI: 10.1016/j.mcm.2010.04.009
[7]Chawla MM, Katti CP. A uniform mesh finite difference method for a class of singular two-point boundary value problems. SIAM Journal on Numerical Analysis, 1985, 22(3), 561-565. DOI: 10.1137/0722033
[8]Roul P, Goura VP, Agarwal R. A compact finite differ-ence method for a general class of nonlinear singular boundary value problems with neumann and robin boundary conditions. Applied Mathematics and Computation, 2019, 350, 283-304. DOI: 10.1016/j.amc.2019.01.001
[9]De Hoog FR, Weiss R. Collocation methods for singular boundary value problems. SIAM Journal on Numerical Analysis, 1978, 15(1), 198-217. DOI: 10.1137/0715013
[10]Roul P, Thula K. A new high-order numerical method for solving singular two-point boundary value problems. Journal of Computational and Applied Mathematics, 2018, 343, 556-574. DOI: 10.1016/j.cam.2018.04.056
[11]Turkyilmazoglu M. Effective computation of exact and analytic approximate solutions to singular nonlinear equations of lane-emden-fowler type. Applied Mathematical Modelling, 2013, 37(14-15), 7539-7548. DOI: 10.1016/j.apm.2013.02.014
[12]Babolian E, Eftekhari A, Saadatmandi A. A sinc-galerkin technique for the numerical solution of a class of singular boundary value problems. Compu-tational and Applied Mathematics, 2015, 34(1), 45-63. DOI: 10.1007/s40314-013-0103-x
[13]Kanth AR, Bhattacharya V. Cubic spline for a class of non-linear singular boundary value problems arising in physiology. Applied Mathematics and Computation, 2006, 174(1), 768-774. DOI: 10.1016/j.amc.2005.05.022
[14]Çağlar H, Çağlar N, Özer M. B-spline solution of non-linear singular boundary value problems arising in physiology. Chaos, Solitons & Fractals, 2009, 39(3), 1232-1237. DOI: 10.1016/j.chaos.2007.06.007
[15]Kanth AR, Aruna K. He’s variational iteration method for treating non-linear singular boundary value problems. Computers & Mathematics with Applications, 2010, 60(3), 821-829. DOI: 10.1016/j.camwa.2010.05.029
[16]Wazwaz AM. The variational iteration method for solving nonlinear singular boundary value problems arising in various physical models. Communications in Nonlinear Science and Numerical Simulation, 2011, 16(10), 3881-3886. DOI: 10.1016/j.cnsns.2011.02.026
[17]Wang F, Salama SA, Khater MM. Optical wave solutions of perturbed time-fractional nonlinear schr¨odinger equation. Journal of Ocean Engineering and Science, 2022. DOI: 10.1016/j.joes.2022.03.014
[18]El-Kalla IL, El Mhlawy AM, Botros M. A continuous solution of solving a class of nonlinear two-point boundary value problem using adomian decomposition method. Ain Shams Engineering Journal, 2019, 10(1), 211-216. DOI: 10.1016/j.asej.2018.11.002
[19]Ebaid A. A new analytical and numerical treatment for singular two-point boundary value problems via the adomian decomposition method. Journal of Computational and Applied Mathematics, 2011, 235(8), 1914-1924. DOI: 10.1016/j.cam.2010.09.007
[20]Umesh, Kumar M. Numerical solution of singular boundary value problems using advanced adomian decomposition method. Engineering with Computers, 2021, 37(4), 2853-2863. DOI: 10.1007/s00366-020-00972-6
[21]Roos HG, Stynes M, Tobiska L. Robust Numerical Methods for Singularly Perturbed Differential Equations: Convection-Diffusion-Reaction and Flow Problems, volume 24. Springer, Berlin Heidelberg, 2008.
[22]O'malley RE. Singular Perturbation Methods for Ordinary Differential Equations, volume 89. Springer, New York, 1991.
[23]Lin B, Li K, Cheng Z. B-spline solution of a singularly perturbed boundary value problem arising in biology. Chaos, Solitons & Fractals, 2009, 42(5), 2934-2948. DOI: 10.1016/j.chaos.2009.04.036
[24]Natesan S, Ramanujam N. Initial-value technique for singularly perturbed boundary-value problems for second-order ordinary differential equations arising in chemical reactor theory. Journal of Optimization Theory and Applications, 1998, 97(2), 455-470. DOI: 10.1023/A:1022639003366
[25]Glizer VY. Asymptotic solution of a boundary-value problem for linear singularly-perturbed functional differential equations arising in optimal control theory. Journal of Optimization Theory and Applications, 2000, 106(2), 309-335. DOI: 10.1023/A:1004651430364
[26]El-Zahar ER, El-Kabeir SM. A new method for solving singularly perturbed boundary value problems. Applied Mathematics & Information Sciences, 2013, 7(3), 927.
[27]Jiwrai R, Mittal RC. A higher order numerical scheme for singularly perturbed burger-huxley equation. Journal of Applied Mathematics & Informatics, 2011, 29(34), 813-829. DOI: 10.14317/jami.2011.29.3_4.813
[28]Kadalbajoo MK, Reddy YN. Initial-value technique for a class of nonlinear singular perturbation problems. Journal of Optimization Theory and Applications, 1987, 53(3), 395-406. DOI: 10.1007/BF00938946
[29]Gasparo MG, Macconi M. New initial-value method for singularly perturbed boundary-value problems. Journal of Optimization Theory and Applications, 1989, 63(2), 213-224. DOI: 10.1007/BF00939575
[30]El-Zahar ER. Approximate analytical solution of singularly perturbed boundary value problems in maple. AIMS Mathematics, 2020, 5(3), 2272-2284.
[31]Gasparo MG, Macconi M. Initial-value methods for second-order singularly perturbed boundary-value problems. Journal of Optimization Theory and Applications, 1990, 66(2), 197-210. DOI: 10.1007/BF00939534
[32]Reddy YN, Chakravarthy PP. An initial-value approach for solving singularly perturbed two-point boundary value problems. Applied Mathematics and Computation, 2004, 155(1), 95-110. DOI: 10.1016/S0096-3003(03)00763-X
[33]Mustafa G, Ejaz ST, Baleanu D, Ghaffar A, Nisar KS. A subdivision-based approach for singularly perturbed boundary value problem. Advances in Difference Equations, 2020, 2020(1), 1-20. DOI: 10.1186/s13662-020-02732-8
[34]Lubuma JM, Patidar KC. Uniformly convergent non-standard finite difference methods for self-adjoint singular perturbation problems. Journal of Computational and Applied Mathematics, 2006, 191(2), 228-238. DOI: 10.1016/j.cam.2005.06.039
[35]Niijima K. On a three-point difference scheme for a singular perturbation problem without a first derivative term i. Memoirs of Numerical Mathematics, 1980, 7, 1-10.
[36]Aziz T, Khan A. A spline method for second-order singularly perturbed boundary-value problems. Journal of Computational and Applied Mathematics, 2002, 147(2), 445-452. DOI: 10.1016/S0377-0427(02)00479-X
[37]Khan I, Aziz T. Tension spline method for second-order singularly perturbed boundary-value problems. International Journal of Computer Mathematics, 2005, 82(12), 1547-1553. DOI: 10.1080/00207160410001684280
[38]Bawa RK, Natesan S. A computational method for self-adjoint singular perturbation problems using quintic spline. Computers & Mathematics with Applications, 2005, 50(8-9), 1371-1382. DOI: 10.1016/j.camwa.2005.04.017
[39]Kumar V, Mehra M. Wavelet optimized finite difference method using interpolating wavelets for self-adjoint singularly perturbed problems. Journal of Computational and Applied Mathematics, 2009, 230(2), 803-812. DOI: 10.1016/j.cam.2009.01.017
[40]Pandit S, Kumar M. Haar wavelet approach for numerical solution of two parameters singularly perturbed boundary value problems. Applied Mathematics & Information Sciences, 2014, 8(6), 2965. DOI: 10.12785/amis/080634
[41]Gadyl’shin RR. Concordance method of asymptotic expansions in a singularly-perturbed boundary-value problem for the laplace operator. Journal of Mathematical Sciences, 2005, 125(5), 579-609. DOI: 10.1007/s10958-005-0062-x
[42]Weinan E, Han J, Jentzen A. Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning. Nonlinearity, 2021, 35(1), 278. DOI: 10.1088/1361-6544/ac337f
[43]Madenci E, Guven I. The finite element method and applications in engineering using ANSYS. Springer, 2015.
[44]E W, Yu B. The deep ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 2018, 6(1), 1-12. DOI: 10.1007/s40304-018-0127-z
[45]JSirignano J, Spiliopoulos K. Dgm: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 2018, 375, 1339-1364. DOI: 10.1016/j.jcp.2018.08.029
[46]Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378, 686-707. DOI: 10.1016/j.jcp.2018.10.045
[47]Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint, 2014. DOI: 10.48550/arXiv.1412.6980
[48]Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 1998, 9(5), 987-1000. DOI: 10.1109/72.712178
[49]Bottou L. Stochastic gradient descent tricks. Neural Networks: Tricks of the Trade, 2nd ed., Springer: Berlin, 2012; pp. 421-436. DOI: 10.1007/978-3-642-35289-8_25
[50]Wang S, Teng Y, Perdikaris P. Understanding and mitigating gradient pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 2021, 43(5), A3055-A3081. DOI: 10.1137/20M1318043
[51]Gao B, Yao R, Li Y. Physics-informed neural networks with adaptive loss weighting algorithm for solving partial differential equations. Computers & Mathematics with Applications, 2025, 181, 216-227. DOI: 10.1016/j.camwa.2025.01.007
[52]Dolean V, Heinlein A, Mishra S, Moseley B. Multilevel domain decomposition-based architectures for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 2024, 429, 117116. DOI: 10.1016/j.cma.2024.117116
[53]Li R, Lee E, Luo T. Physics-informed neural networks for solving multiscale mode-resolved phonon Boltzmann transport equation. Materials Today Physics, 2021, 19, 100429. DOI: 10.1016/j.mtphys.2021.100429
[54]Fang Z, Wang S, Perdikaris P. Ensemble learning for physics informed neural networks: A gradient boosting approach. arXiv preprint arXiv:2302.13143, 2023. DOI: 10.48550/arXiv.2302.13143
[55]Martinek P, Krammer O. Optimising pin-in-paste technology using gradient boosted decision trees. Soldering & Surface Mount Technology, 2018, 30(3), 164-170. DOI: 10.1108/SSMT-09-2017-0024
[56]Jagtap AD, Kawaguchi K, Em Karniadakis G. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proceedings of the National Academy of Sciences, 2020, 476(2239), 21948-21956. DOI: 10.1098/rspa.2020.0334
[57]Jagtap AD, Kawaguchi K, Karniadakis GE. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics, 2020, 404, 109136. DOI: 10.1016/j.jcp.2019.109136
[58]Botev A, Ritter H, Barber D. Practical gauss-newton optimization for deep learning. InInternational Conference on Machine Learning, 2017, 70, 557-565.
[59]Hao W, Hong Q, Jin X. Gauss Newton method for solving variational problems of PDEs with neural network discretizaitons. Journal of Scientific Computing, 2024, 100(1), 17. DOI: 10.1007/s10915-024-02535-z
[60]Gratton S, Mercier V, Riccietti E, Toint PL. A block-coordinate approach of multi-level optimization with an application to physics-informed neural networks. Computational Optimization and Applications, 2024, 89(2), 385-417. DOI: 10.1007/s10589-024-00597-1
[61]Golub G, Pereyra V. Separable nonlinear least squares: the variable projection method and its applications. Inverse Problems, 2003, 19(2). DOI: 10.1088/0266-5611/19/2/201
[62]Kim CT, Lee JJ. Training two-layered feedforward networks with variable projection method. IEEE Transactions on Neural Networks, 2008, 19(2),371-375. DOI: 10.1109/TNN.2007.911739
[63]Dong S, Yang J. Numerical approximation of partial differential equations by a variable projection method with artificial neural networks. Computer Methods in Applied Mechanics and Engineering, 2022, 398, 115284. DOI: 10.1016/j.cma.2022.115284
[64]Huang X, Liu H, Shi B, Wang Z, Yang K, et al. A Universal PINNs Method for Solving Partial Differential Equations with a Point Source, International Joint Conferences on Artificial Intelligence, 2022, 3839-3846. DOI: 10.24963/ijcai.2022/533
[65]Zubov K, McCarthy Z, Ma Y, Calisto F, Pagliarino V, et al. Neuralpde: Automating physics-informed neural networks (pinns) with error approximations. arXiv preprint arXiv:2107.09443, 2021. DOI: 10.48550/arXiv.2107.09443
[66]Nash SG, Nocedal J. A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization. SIAM Journal on Optimization, 1991, 1(3), 358-372. DOI: 10.1137/0801023
[67]Nocedal J. Updating quasi-newton matrices with limited storage. Mathematics of Computation, 1980, 35(151), 773-782.
[68]Roul P, Goura VP. A fast numerical scheme for solving singular boundary value problems arising in various physical models. Journal of Mathematical Chemistry, 2022, 60(3), 514-541. DOI: 10.1007/s10910-021-01316-5
[69]Roul P, Kumari T. A quartic trigonometric B-spline collocation method for a general class of nonlinear singular boundary value problems. Journal of Mathematical Chemistry, 2022, 1-17. DOI: 10.1007/s10910-021-01293-9
[70]Natesan S, Ramanujam N. A “booster method” for singular perturbation problems arising in chemical reactor theory. Applied mathematics and computation, 1999, 100(1), 27-48. DOI: 10.1016/S0096-3003(98)00014-9
[71]Celik E, Tunc H, Sari M. An efficient multi-derivative numerical method for chemical boundary value problems. Journal of Mathematical Chemistry, 2024, 62(3), 634-653. DOI: 10.1007/s10910-023-01556-7
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Global Integrated Mathematics
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
