In recent years, some authors have used neural networks (ANNs) as an important technique to solve many real-world problems because of their specifications. Some authors have used ANNs to solve different types of differential equations, such that ^{1, 3} first proposed the concept of solving differential equations using ANNs by formulating a trial solution of the differential equation. The authors tested the applicability and accuracy of their developed method not only for differential equations but also for systems of coupled differential equations. Furthermore, the authors compared their results with those obtained using other numerical methods and reported that the developed ANN was superior in terms of memory requirements and accuracy. ^{4– 6} For this reason, the authors aimed to develop this technique to obtain the best results. One of these developments is the training rules, particularly the quasi-Newton method, because it is a second-order convergence. Many authors such ^{7– 12} have proposed modifications for the training algorithm. Others such ^{13– 20} suggest some rules for the speed of convergence. Several attempts have been made to solve different types of differential equations by using feed forward neural networks. In, ²¹ reported a hybrid method was reported that combines optimization techniques with neural networks to solve high-order differential equations.

The quasi-Newton method is the most useful method for minimizing a smooth n variable function. minimize f ( x ) , x ∈ R n (1) where f : R n → R 1 is continuously differentiable. ²² In contrast to utilizing the real value of the Hessian or its inverse, in the proposed update, we use a symmetric positive definite estimate of the Hessian (H) or its inverse ( inv H). The following is the form: x k + 1 = x k + α k d k , d k = − g k H k = − g k H k − 1 (2)

If H is not an invertible matrix, then the pseudoinverse of H.

Wolfe conditions are used to determine the step length ( α k ) and search direction ( d k ), as follows: f ( x k + α k d k ) ≤ f ( x k ) + δ α k g k T d k (3) d k T g ( x k + α k d k ) ≥ σ d k T g k (4) where 0 < δ < σ < 1 was typically used. For more details, refer to. ²³ The parameter α k is computed using a line - search in the following form: α k = − g k T d k / d k T Q d k (5) For more details, please refer to. ²⁴ Its direction is computed by solving: B k d k + g k = 0 (6)

For each iteration, B k is the updated Hessian estimate. The Broyden Fletcher Goldfarb-Shanno (BFGS) approach, proposed by Broyden, Fletcher, Goldfarb, and Shanno, is now one of the most effective training methods. Using the following formula, matrix B k + 1 in the BFGS technique can be updated: B k + 1 BFGS = B k − B k s k s k T B k T s k T B k s k + y k y k T s k T y k (7)

Let H k be the inverse of B k . Undoubtedly, the suggested update in (8) is publicly known as H k + 1 BFGS = H k − H k y k s k T + s k y k T H k s k T y k + [ 1 + y k T H k y k s k T y k ] s k s k T s k T y k (8)

See ^{25, 26} for further details. For the update process, we let: B k + 1 s k = y k (9) where s k = x k + 1 − x k = α k d k and y k = g k + 1 − g k (see ²⁷ ). The numerical experiment showed that the BFGS technique outperformed all the other training approaches. Convex minimization using the update approach has been extensively investigated; for example, see. ^{1, 2, 28} To demonstrate that the update approach using the Wolfe line search may not succeed for non-convex functions, Dai created an example with six cycling points. ²⁹ Many improvements have been suggested, including changes in the regular BFGS technique, and a modified BFGS algorithm (MBFGS) has been devised to improve and speed the global convergence of the BFGS method. ^{30, 31} They demonstrated that the approach converged worldwide for nonconvex optimization problems. To determine whether a novel quasi-Newton methodology has global convergence and outperforms the BFGS method in terms of computation, see. ^{32, 33} In practice, the modified BFGS technique is typically preferred to efficiently compute matrix H (or H ⁻¹) using a symmetric positive definite matrix. While the standard method employs only gradient values, the modified approach uses both. Without making any convexity assumptions about the goal function, global convergence was demonstrated. ³⁴

A new additional update was derived using a quadratic model of the goal function. Consequently, the quadratic model of the objective function is given as f k + 1 = f k + s k T g k + 1 2 s k T Q ( x k ) s k (10) where Q ( x k ) is the Hessian matrix. The first derivative of the above equation can be written as: ∇ f k + 1 = g k + Q ( x k ) s k (11)

Thus, the curvature information in Eq. (10) can be approximated by s k T Q ( x k ) s k = 2 3 ( f k − f k + 1 ) + 2 3 s k T Q ( x k ) s k (12)

Because the updated B k + 1 is supposed to approximate the Q ( x k ) , it is reasonable to have s k T B k + 1 s k = 2 3 ( f k − f k + 1 ) + 2 3 s k T Q ( x k ) s k (13)

Using (11) in (13), we obtain: s k T B k + 1 s k = 2 3 s k T y k + 2 3 ( f k − f k + 1 ) (14)

The new quasi-Newton (QN-) equation is given by: s k T y k ~ = 2 3 s k T y k + 2 3 ( f k − f k + 1 ) (15)

From the above equation, the different gradients can be written as B k + 1 s k = y k ~ , y k ~ = 2 3 y k + 2 / 3 ( f k − f k + 1 ) s k T u k u k (16) where u k is a vector such that s k T u k ≠ 0 . The BFGS update is modified based on the revised quasi-Newton equation. Alternatively, the vector u k choices in Equation (16) can be expressed as: (i)

For u k = y k , Equation (16) becomes:

From the above explanation of the results, we can write the algorithm as follows:

Stage 1: Let x 0 ∈ R n , k = 0 and H 0 = I

Stage 2: If ‖ g k ‖ = 0 , stop.

Stage 3: Evaluate d k = − H k g k .

Stage 4: Determine the optimal learning rate (step - size) by α k using Eqs. (4) & (5).

Stage 5: Let x k + 1 = x k + α k d k . Update H k + 1 by using Equations (9) and (16) if s k T y k ~ > 0 ; otherwise, leave H k + 1 = H k .

Stage 6: Take k = k + 1 , and then go to Stage 2.

The following theorem illustrates the theoretical benefits of the new quasi-Newton Equation (16). To ensure that the matrix B k + 1 is positive definite, we need only prove that s k T y k ~ > 0 holds. Theorem 1.

Let matrix sequence B k + 1 be generated using Equation (6). Thus, the sequence B k + 1 is positive- definite.

We provide a global convergence of innovative approaches under circumstances that are comparatively understated. 1.

The level was set to L 0 = { x ∈ R n : f ( x ) ≤ f ( x 0 ) } be convex.

The series { x k } generated by a new algorithm is evident in S because { f k } is a decreasing series, and there is a constant f ∗ that results in lim k → ∞ f k = f ∗ (21) 3.

Let Q be a matrix from the 2 ^nd derivatives of the f . Then, there exist constants R and r , such that:

for all z ∈ R n , for more details see. ^{12– 14} Theorem 2.

If { x k } is generated using the proposed algorithm. Then we have: r ‖ s k ‖ 2 ≤ s k T y k ~ ≤ R ‖ s k ‖ 2 . (23) and ‖ y k ~ ‖ ≤ ( L + R ) ‖ s k ‖ . (24)

In this section, we present a numerical comparison of QN -techniques and suggest modifications for solving 4 ^th order nonlinear partial differential equations. Example 1:

Consider the nonlinear 4 ^th order has the form; ʯ xt − ʯ xxxy − 2 ʯ xx ʯ y − 4 ʯ x ʯ xy = 0 ; ʯ ( x , y , 0 ) = 1 2 sech 2 ( 1 2 ( x + y ) ) and , exact solution ʯ ( x , y , z , t ) = tanh ( 1 2 ( x + y − t ) )

The results of solving the above equation at different times t are presented in Table 1. The neural solution for this equation is shown in Figure 1.

We stopped utilizing the algorithms by employing Himmeblau's law ¹⁸ :

If | f ( x k ) | > 1 0 − 5 , then | f ( x k ) − f ( x k + 1 ) | | f ( x k ) | = 1 . Otherwise, | f ( x k ) − f ( x k + 1 ) | = 1 . For every problem, if ‖ g k ‖ < ε is used, the program is terminated.

Quasi-Newton approaches perform better when an appropriate quasi-Newton equation is employed. The performance of the new update with u k = g k + 1 was the best of the three methods, whereas the normal performance of the new update with u k = y k and u k = g k was somewhat better than that of the BFGS technique. As a result, among the QN -procedures for unconstrained problems, the new update with u k = g k + 1 is the most well -organized.

In this study, we constructed improved BFGS quasi-Newton updating formulae by using the proposed robust QN -equation. Second-order information from Hessian’s Hessian objective function Hessian’s is used in this study to develop a novel quasi-Newton equation. Two nonlinear 4 ^th order example are provided to illustrate the accuracy of the suggested update, The results are consistent with the practical results and conform to the results that the suggested update, is globally convergent.