Advanced Physics-Informed Neural Network with Residuals for Solving Complex Integral Equations

Document Type : Research Article

Authors

1 Department of Computer and Data Sciences, Faculty of Mathematical Sciences, Shahid Beheshti University, Tehran, Iran

2 Department of Computer and Data Sciences, Faculty of Mathematical Sciences, Shahid Beheshti University, Tehran, Iran; Department of Cognitive Modeling, Institute for Cognitive and Brain Sciences, Shahid Beheshti University

Abstract

In this paper, we present the Residual Integral Solver Network (RISN), a novel neural network architecture designed to solve a wide range of integral and integro-differential equations, including one-dimensional, multi-dimensional, ordinary and partial integro-differential, systems, fractional types, and Helmholtz-type integral equations involving oscillatory kernels. RISN integrates residual connections with high-accuracy numerical methods such as Gaussian quadrature and fractional derivative operational matrices, enabling it to achieve higher accuracy and stability than traditional Physics-Informed Neural Networks (PINN). The residual connections help mitigate vanishing gradient issues, allowing RISN to handle deeper networks and more complex kernels, particularly in multi-dimensional problems. Through extensive experiments, we demonstrate that RISN consistently outperforms not only classical PINNs but also advanced variants such as Auxiliary PINN (A-PINN) and Self-Adaptive PINN (SA-PINN), achieving significantly lower Mean Absolute Errors (MAE) across various types of equations. These results highlight RISN’s robustness and efficiency in solving challenging integral and integro-differential problems, making it a valuable tool for real-world applications where traditional methods often struggle.

Keywords

Main Subjects

Full Text

 

Article PDF

References

[1] A.-M. Wazwaz, Linear and nonlinear integral equations, vol. 639. Springer, 2011.
[2] R. Kress, Linear integral equations, (1989).
[3] K. G. TeBeest, Classroom note: numerical and analytical solutions of volterra’s population model, SIAM review, 39(3), 484–493, (1997).
[4] W. O. Kermack and A. G. McKendrick, A contribution to the mathematical theory of epidemics, Proceedings of the royal society of london. Series A, Containing papers of a mathematical and physical character, 115(772), 700–721, (1927).
[5] R. Estrada and R. P. Kanwal, Singular integral equations. Springer Science & Business Media, 2012.
[6] F. M. Scudo, Vito volterra and theoretical ecology, Theoretical population biology, 2(1), 1–23, (1971).
[7] A. A. Minakov and C. Schick, Non-equilibrium fast thermal response of polymers, 660, 82–93, (2018).
[8] S. A. Khuri and A.-M. Wazwaz, The decomposition method for solving a second kind fredholm integral equation with a logarithmic kernel, International journal of computer mathematics, 61(1-2), 103–110, (1996).
[9] J. W. Thomas, Numerical partial differential equations: finite difference methods, vol. 22. Springer Science & Business Media, 2013.
[10] M. Dehghan, Finite difference procedures for solving a problem arising in modeling and design of certain optoelectronic devices, Mathematics and Computers in Simulation, 71(1), 16–30, (2006).
[11] O. C. Zienkiewicz and R. L. Taylor, The finite element method for solid and structural mechanics. Elsevier, 2005.
[12] S. David Müzel, E. P. Bonhin, N. M. Guimarães, and E. S. Guidi, Application of the finite element method in the analysis of composite materials: A review, Polymers, 12(4), 818, (2020).
[13] J. P. Boyd, Chebyshev and Fourier spectral methods. Courier Corporation, 2001.
[14] F. S. Yousefi, Y. Ordokhani, and S. Yousefi, Numerical solution of variable order fractional differential equations by using shifted legendre cardinal functions and ritz method, Engineering with Computers, 38(3), 1977–1984, (2022).
[15] S. Latifi, K. Parand, and M. Delkhosh, Generalized lagrange–jacobi–gauss–radau collocation method for solving a nonlinear optimal control problem with the classical diffusion equation, 135, 1–19, (2020).
[16] M. Griebel and M. Holtz, Dimension-wise integration of high-dimensional functions with applications to finance, Journal of Complexity, 26(5), 455–489, (2010).
[17] P. Bastian, M. Blatt, A. Dedner, C. Engwer, R. Klöfkorn, R. Kornhuber, M. Ohlberger, and O. Sander, A generic grid interface for parallel and adaptive scientific computing. part ii: implementation and tests in dune, 82, 121–138, (2008).
[18] P. G. Ciarlet, The finite element method for elliptic problems. SIAM, 2002.
[19] H. Sundar, G. Biros, C. Burstedde, J. Rudi, O. Ghattas, and G. Stadler, Parallel geometric-algebraic multigrid on unstructured forests of octrees, in SC’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 1–11, IEEE, 2012.
[20] V. E. Tarasov, Fractional dynamics: applications of fractional calculus to dynamics of particles, fields and media. Springer Science & Business Media, 2011.
[21] K. Atkinson, An introduction to numerical analysis. John wiley & sons, 1991.
[22] A. Saadatmandi and M. Dehghan, A legendre collocation method for fractional integro-differential equations, Journal of Vibration and Control, 17(13), 2050–2058, (2011).
[23] I. Goodfellow, Deep learning, (2016).
[24] Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, nature, 521(7553), 436–444, (2015).
[25] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, 378, 686–707, (2019).
[26] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nature Reviews Physics, 3(6), 422–440, (2021).
[27] L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, Deepxde: A deep learning library for solving differential equations, SIAM review, 63(1), 208–228, (2021).
[28] M. Raissi and G. E. Karniadakis, Hidden physics models: Machine learning of nonlinear partial differential equations, 357, 125–141, (2018).|
[29] Y. Zhu, Data-Driven and Physics-Constrained Deep Learning for Surrogate Modeling and Uncertainty Quantification of Physical Systems. University of Notre Dame, 2019.
[30] J. Berg and K. Nyström, A unified deep artificial neural network approach to partial differential equations in complex geometries, 317, 28–41, (2018).
[31] J. Sirignano and K. Spiliopoulos, Dgm: A deep learning algorithm for solving partial differential equations, 375, 1339–1364, (2018).
[32] J. Han, A. Jentzen, and W. E, Solving high-dimensional partial differential equations using deep learning, Proceedings of the National Academy of Sciences, 115(34), 8505–8510, (2018).
[33] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778, 2016.
[34] Y. Bengio, P. Simard, and P. Frasconi, Learning long-term dependencies with gradient descent is difficult, IEEE transactions on neural networks, 5(2), 157–166, (1994).
[35] S. Hochreiter, The vanishing gradient problem during learning recurrent neural nets and problem solutions, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02), 107–116, (1998).
[36] D. Gazoulis, I. Gkanis, and C. G. Makridakis, On the stability and convergence of physics informed neural networks, arXiv preprint arXiv:2308.05423, (2023).
[37] L. Yuan, Y.-Q. Ni, X.-Y. Deng, and S. Hao, A-pinn: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations, 462, 111260, (2022).
[38] L. D. McClenny and U. M. Braga-Neto, Self-adaptive physics-informed neural networks, 474, 111722, (2023).
[39] A. A. Aghaei, M. M. Moghaddam, and K. Parand, Pinnies: An efficient physics-informed neural network framework to integral operator problems, (2024).
[40] X. Wang, R. A. Wildman, D. S. Weile, and P. Monk, A finite difference delay modeling approach to the discretization of the time domain integral equations of electromagnetics, IEEE Transactions on Antennas and Propagation, 56(8), 2442–2452, (2008).
[41] R. Brambilla, F. Grilli, L. Martini, and F. Sirois, Integral equations for the current density in thin conductors and their solution by the finite-element method, Superconductor Science and Technology, 21(10), 105008, (2008).
[42] U. Vandandoo, T. Zhanlav, O. Chuluunbaatar, A. Gusev, S. Vinitsky, and G. Chuluunbaatar, High-Order Finite Difference and Finite Element Methods for Solving Some Partial Differential Equations. Springer, 2024.
[43] B. F. Hamfeldt and J. Lesniewski, Convergent finite difference methods for fully nonlinear elliptic equations in three dimensions, Journal of Scientific Computing, 90(1), 35, Dec 2021.
[44] W. Sarakorn and C. Vachiratienchai, Hybrid finite difference–finite element method to incorporate topography and bathymetry for two-dimensional magnetotelluric modeling, Earth, Planets and Space, 70(1), 103, Jun 2018.
[45] W. K. Liu, S. Li, and H. S. Park, Eighty years of the finite element method: Birth, evolution, and future, Archives of Computational Methods in Engineering, 29(6), 4431–4453, Oct 2022.
[46] D. Nath, Ankit, D. R. Neog, and S. S. Gautam, Application of machine learning and deep learning in finite element analysis: A comprehensive review, Archives of Computational Methods in Engineering, 31(5), 2945–2984, Jul 2024.
[47] X. Chen, K. Zhang, Z. Ji, X. Shen, P. Liu, L. Zhang, J. Wang, and J. Yao, Progress and challenges of integrated machine learning and traditional numerical algorithms: Taking reservoir numerical simulation as an example, Mathematics, 11(21), (2023).
[48] S. Effati and R. Buzhabadi, A neural network approach for solving fredholm integral equations of the second kind, 21, 843–852, (2012).
[49] M. Mosleh, Numerical solution of fuzzy linear fredholm integro-differential equation by\\fuzzy neural network, Iranian journal of fuzzy systems, 11(1), 91–112, (2014).
[50] A. Jafarian, S. Measoomy, and S. Abbasbandy, Artificial neural networks based modeling for solving volterra integral equations system, 27, 391–398, (2015).
[51] B. Asady, F. Hakimzadegan, and R. Nazarlue, Utilizing artificial neural network approach for solving two-dimensional integral equations, 8, 1–9, (2014).
[52] S. S. Chaharborj, S. Chaharborj, and Y. Mahmoudi, Study of fractional order integro-differential equations by using chebyshev neural network, J. Math. Stat, 13(1), 1–13, (2017).
[53] R. Guo, T. Shan, X. Song, M. Li, F. Yang, S. Xu, and A. Abubakar, Physics embedded deep neural network for solving volume integral equation: 2-d case, IEEE Transactions on Antennas and Propagation, 70(8), 6135–6147, (2021).
[54] L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3(3), 218–229, Mar 2021.
[55] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Fourier neural operator for parametric partial differential equations, (2021).
[56] M. M. Moghaddam, H. Dana Mazraeh, and K. Parand, A double deep q network-powered approach to discovering symbolic solutions for nonlinear integral equations, 169, 114113, (2026).
[57] M. Movahedian Moghaddam, H. Dana Mazraeh, and K. Parand, Symbolic solution of fractional volterra population models via physics-informed neural networks and distributed particle swarm optimization, 189, 114512, (2026).
[58] H. Dana Mazraeh and K. Parand, Approximate symbolic solutions to differential equations using a novel combination of monte carlo tree search and physics-informed neural networks approach, Engineering with Computers, 41(4), 2707–2735, Aug 2025.
[59] H. Dana Mazraeh and K. Parand, Gels-svr: a combination of grammatical evolution and least squares support vector regression for symbolic solution to falkner-skan equation, Genetic Programming and Evolvable Machines, 27(1), 3, Dec 2025.
[60] H. Dana Mazraeh and K. Parand, A three-stage framework combining neural networks and monte carlo tree search for approximating analytical solutions to the thomasfermi equation, 87, 102582, (2025).
[61] H. Dana Mazraeh and K. Parand, An innovative combination of deep q-networks and context-free grammars for symbolic solutions to differential equations, 142, 109733, (2025).
[62] A. L. Martire, M. A. Congedo, and M. Cenci, Fractional Volterra Integral Equations: A Neural Network Approach, vol. 1. Roma TrE-Press, 2022.
[63] R. Saneifard, A. Jafarian, N. Ghalami, and S. M. Nia, Extended artificial neural networks approach for solving two-dimensional fractional-order volterra-type integro-differential equations, 612, 887897, Sept. 2022.
[64] K. Hornik, M. Stinchcombe, and H. White, Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural networks, 3(5), 551–560, (1990).
[65] A. Pinkus, Approximation theory of the mlp model in neural networks, 8, 143–195, (1999).
[66] H. N. Mhaskar, Neural networks for optimal approximation of smooth and analytic functions, Neural computation, 8(1), 164–177, (1996). 
Analytical and Numerical Solutions for Nonlinear Equations
Volume 11, Issue 1
June 2026
Pages 153-173

Files

History

  • Receive Date: 18 February 2026
  • Revise Date: 16 April 2026
  • Accept Date: 02 June 2026
  • Publish Date: 07 June 2026

Share

How to cite

Statistics