Update Quasi-Newton Algorithm for Training ANN

Farah Ghazi; Luma Tawfiq; Zainab Kareem

doi:10.12688/f1000research.172826.1

Home Browse Update Quasi-Newton Algorithm for Training ANN

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Update Quasi-Newton Algorithm for Training ANN

[version 1; peer review: awaiting peer review]

Farah Ghazi¹, Luma Tawfiq ¹, Zainab Kareem²

PUBLISHED 16 Jan 2026

Author details Author details

¹ Mathematics, University of Baghdad, Baghdad, Iraq
² Ministry of Education, Directorate General of Education, KARKH II, Baghdad, Iraq

Farah Ghazi
Roles: Data Curation, Resources, Software, Validation

Luma Tawfiq
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Review & Editing

Zainab Kareem
Roles: Methodology, Project Administration, Visualization, Writing – Original Draft Preparation

OPEN PEER REVIEW

REVIEWER STATUS AWAITING PEER REVIEW

This article is included in the Fallujah Multidisciplinary Science and Innovation gateway.

Abstract

The proposed design of neural network in this article is based on new accurate approach for training by unconstrained optimization, especially update quasi-Newton methods are perhaps the most popular general-purpose algorithms. A limited memory BFGS algorithm is presented for solving large-scale symmetric nonlinear equations, where a line search technique without derivative information is used. On each iteration, the updated approximations of Hessian matrix satisfy the quasi-Newton form, which traditionally served as the basis for quasi-Newton methods. On the basis of the quadratic model used in this article, we add a new update of quasi-Newton form. One innovative features of this form's is its ability to estimate the energy function's or performance function with high order precision with second-order curvature while employ the given function value data and gradient. The global convergence of the proposed algorithm is established under some suitable conditions. Under some hypothesis the approach is established to be globally convergent. The updated approaches can be numerical and more efficient than the existing comparable traditional methods, as illustrated by numerical trials. Numerical results show that the given method is competitive to those of the normal BFGS methods. We show that solving a partial differential equation can be formulated as a multi-objective optimization problem, and use this formulation to propose several modifications to existing methods. Also the proposed algorithm is used to approximate the optimal scaling parameter, which can be used to eliminate the need to optimize this parameter. Our proposed update is tested on a variety of partial differential equations and compared to existing methods. These partial differential equations include the fourth order three dimensions nonlinear equation, which we solve in up to four dimensions, the convection-diffusion equation, all of which show that our proposed update lead to enhanced accuracy.

Keywords

Robust quasi-Newton methods, Convergence analysis, Numerical experiments, ANNs. unconstrained optimization.

Corresponding author: Luma Tawfiq

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2026 Ghazi F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The author(s) is/are employees of the US Government and therefore domestic copyright protection in USA does not apply to this work. The work may be protected under the copyright laws of other jurisdictions when used in those jurisdictions.

How to cite: Ghazi F, Tawfiq L and Kareem Z. Update Quasi-Newton Algorithm for Training ANN [version 1; peer review: awaiting peer review]. F1000Research 2026, 15:71 (https://doi.org/10.12688/f1000research.172826.1) First published: 16 Jan 2026, 15:71 (https://doi.org/10.12688/f1000research.172826.1) Latest published: 16 Jan 2026, 15:71 (https://doi.org/10.12688/f1000research.172826.1)

1. Introduction

In recent years, some authors have used neural networks (ANNs) as an important technique to solve many real-world problems because of their specifications. Some authors have used ANNs to solve different types of differential equations, such that^1,3 first proposed the concept of solving differential equations using ANNs by formulating a trial solution of the differential equation. The authors tested the applicability and accuracy of their developed method not only for differential equations but also for systems of coupled differential equations. Furthermore, the authors compared their results with those obtained using other numerical methods and reported that the developed ANN was superior in terms of memory requirements and accuracy.^4–6 For this reason, the authors aimed to develop this technique to obtain the best results. One of these developments is the training rules, particularly the quasi-Newton method, because it is a second-order convergence. Many authors such^7–12 have proposed modifications for the training algorithm. Others such^13–20 suggest some rules for the speed of convergence. Several attempts have been made to solve different types of differential equations by using feed forward neural networks. In,²¹ reported a hybrid method was reported that combines optimization techniques with neural networks to solve high-order differential equations.

The quasi-Newton method is the most useful method for minimizing a smooth n variable function.

(1)

minimize f (x), x \in R^{n}

where

f : R^{n} \to R^{1}

is continuously differentiable.²² In contrast to utilizing the real value of the Hessian or its inverse, in the proposed update, we use a symmetric positive definite estimate of the Hessian (H) or its inverse (inv H). The following is the form:

(2)

x_{k + 1} = x_{k} + α_{k} d_{k}, d_{k} = - \frac{g_{k}}{H_{k}} = - g_{k} H_{k}^{- 1}

If H is not an invertible matrix, then the pseudoinverse of H.

Wolfe conditions are used to determine the step length ( $α_{k}$ ) and search direction ( $d_{k}$ ), as follows:

(3)

f (x_{k} + α_{k} d_{k}) \leq f (x_{k}) + δ α_{k} g_{k}^{T} d_{k}

(4)

d_{k}^{T} g (x_{k} + α_{k} d_{k}) \geq σ d_{k}^{T} g_{k}

where

0 < δ < σ < 1

was typically used. For more details, refer to.²³ The parameter

α_{k}

is computed using a line - search in the following form:

(5)

α_{k} = - g_{k}^{T} d_{k} / d_{k}^{T} Q d_{k}

For more details, please refer to.²⁴ Its direction is computed by solving:

(6)

B_{k} d_{k} + g_{k} = 0

For each iteration, $B_{k}$ is the updated Hessian estimate. The Broyden Fletcher Goldfarb-Shanno (BFGS) approach, proposed by Broyden, Fletcher, Goldfarb, and Shanno, is now one of the most effective training methods. Using the following formula, matrix $B_{k + 1}$ in the BFGS technique can be updated:

(7)

B_{k + 1}^{BFGS} = B_{k} - \frac{B_{k} s_{k} s_{k}^{T} {B_{k}}^{T}}{s_{k}^{T} B_{k} s_{k}} + \frac{y_{k} y_{k}^{T}}{s_{k}^{T} y_{k}}

Let $H_{k}$ be the inverse of $B_{k}$ . Undoubtedly, the suggested update in (8) is publicly known as

(8)

H_{k + 1}^{BFGS} = H_{k} - \frac{H_{k} y_{k} s_{k}^{T} + s_{k} y_{k}^{T} H_{k}}{s_{k}^{T} y_{k}} + [1 + \frac{y_{k}^{T} H_{k} y_{k}}{s_{k}^{T} y_{k}}] \frac{s_{k} s_{k}^{T}}{s_{k}^{T} y_{k}}

See^25,26 for further details. For the update process, we let:

(9)

B_{k + 1} s_{k} = y_{k}

where

s_{k} = x_{k + 1} - x_{k} = α_{k} d_{k}

and

y_{k} = g_{k + 1} - g_{k}

(see²⁷). The numerical experiment showed that the BFGS technique outperformed all the other training approaches. Convex minimization using the update approach has been extensively investigated; for example, see.^1,2,28 To demonstrate that the update approach using the Wolfe line search may not succeed for non-convex functions, Dai created an example with six cycling points.²⁹ Many improvements have been suggested, including changes in the regular BFGS technique, and a modified BFGS algorithm (MBFGS) has been devised to improve and speed the global convergence of the BFGS method.^30,31 They demonstrated that the approach converged worldwide for nonconvex optimization problems. To determine whether a novel quasi-Newton methodology has global convergence and outperforms the BFGS method in terms of computation, see.^32,33 In practice, the modified BFGS technique is typically preferred to efficiently compute matrix H (or H⁻¹) using a symmetric positive definite matrix. While the standard method employs only gradient values, the modified approach uses both. Without making any convexity assumptions about the goal function, global convergence was demonstrated.³⁴

2. Derivation of suggested update

A new additional update was derived using a quadratic model of the goal function. Consequently, the quadratic model of the objective function is given as

(10)

f_{k + 1} = f_{k} + s_{k}^{T} g_{k} + \frac{1}{2} s_{k}^{T} Q (x_{k}) s_{k}

where

Q (x_{k})

is the Hessian matrix. The first derivative of the above equation can be written as:

(11)

\nabla f_{k + 1} = g_{k} + Q (x_{k}) s_{k}

Thus, the curvature information in Eq. (10) can be approximated by

(12)

s_{k}^{T} Q (x_{k}) s_{k} = \frac{2}{3} (f_{k} - f_{k + 1}) + \frac{2}{3} s_{k}^{T} Q (x_{k}) s_{k}

Because the updated $B_{k + 1}$ is supposed to approximate the $Q (x_{k})$ , it is reasonable to have

(13)

s_{k}^{T} B_{k + 1} s_{k} = \frac{2}{3} (f_{k} - f_{k + 1}) + \frac{2}{3} s_{k}^{T} Q (x_{k}) s_{k}

Using (11) in (13), we obtain:

(14)

s_{k}^{T} B_{k + 1} s_{k} = \frac{2}{3} s_{k}^{T} y_{k} + \frac{2}{3} (f_{k} - f_{k + 1})

The new quasi-Newton (QN-) equation is given by:

(15)

s_{k}^{T} \tilde{y_{k}} = \frac{2}{3} s_{k}^{T} y_{k} + \frac{2}{3} (f_{k} - f_{k + 1})

From the above equation, the different gradients can be written as

(16)

B_{k + 1} s_{k} = \tilde{y_{k}}, \tilde{y_{k}} = \frac{2}{3} y_{k} + \frac{2 / 3 (f_{k} - f_{k + 1})}{s_{k}^{T} u_{k}} u_{k}

where

u_{k}

is a vector such that

s_{k}^{T} u_{k} \neq 0

. The BFGS update is modified based on the revised quasi-Newton equation. Alternatively, the vector

u_{k}

choices in Equation (16) can be expressed as:

(i) For $u_{k} = y_{k}$ , Equation (16) becomes:

\tilde{y_{k}} = \frac{2}{3} y_{k} + \frac{2 / 3 (f_{k} - f_{k + 1})}{s_{k}^{T} y_{k}} y_{k} .

(ii) For $u_{k} = g_{k}$ , Equation (16) becomes:

\tilde{y_{k}} = \frac{2}{3} y_{k} + \frac{2 / 3 (f_{k} - f_{k + 1})}{s_{k}^{T} g_{k}} g_{k} .

(iii) For $u_{k} = g_{k + 1}$ , Equation (16) becomes:

\tilde{y_{k}} = \frac{2}{3} y_{k} + \frac{2 / 3 (f_{k} - f_{k + 1})}{s_{k}^{T} g_{k + ` 1}} g_{k + ` 1} .

From the above explanation of the results, we can write the algorithm as follows:

Stage 1: Let $x_{0} \in R^{n}$ , $k = 0$ and $H_{0} = I$

Stage 2: If $‖ g_{k} ‖ = 0$ , stop.

Stage 3: Evaluate $d_{k} = - H_{k} g_{k}$ .

Stage 4: Determine the optimal learning rate (step - size) by $α_{k}$ using Eqs. (4) & (5).

Stage 5: Let $x_{k + 1} = x_{k} + α_{k} d_{k}$ . Update $H_{k + 1}$ by using Equations (9) and (16) if $s_{k}^{T} \tilde{y_{k}} > 0$ ; otherwise, leave $H_{k + 1} = H_{k}$ .

Stage 6: Take $k = k + 1$ , and then go to Stage 2.

The following theorem illustrates the theoretical benefits of the new quasi-Newton Equation (16). To ensure that the matrix $B_{k + 1}$ is positive definite, we need only prove that $s_{k}^{T} \tilde{y_{k}} > 0$ holds.

Theorem 1.

Let matrix sequence $B_{k + 1}$ be generated using Equation (6). Thus, the sequence $B_{k + 1}$ is positive- definite.

Proof.

From the different gradient definitions, we have:

(17)

s_{k}^{T} \tilde{y_{k}} = \frac{2}{3} y_{k} + \frac{2}{3} (f_{k} - f_{k + 1})

By applying Wolfe's condition to the previous equation, we obtain:

(18)

s_{k}^{T} \tilde{y_{k}} \geq \frac{2}{3} (s_{k}^{T} y_{k} - δ g_{k}^{T} s_{k})

Because $s_{k}^{T} y_{k} > 0$ and $- δ g_{k}^{T} s_{k} > 0$ , Eq. (18), we obtain

(19)

s_{k}^{T} \tilde{y_{k}} \geq 0

Therefore, $B_{k + 1}$ is positive -definite.

3. Convergent analysis

We provide a global convergence of innovative approaches under circumstances that are comparatively understated.

1. The level was set to $L_{0} = {x \in R^{n} : f (x) \leq f (x_{0})}$ be convex.
2. Because the gradient satisfies the Lipschitz continuity, there is a positive constant called $L > 0$ :

(20)

(\nabla f (\bar{x}) - \nabla f (x^{+})) \leq L ‖ \bar{x} - x^{+} ‖, \forall \bar{x}, x^{+} \in L_{0} .

The series ${x_{k}}$ generated by a new algorithm is evident in $S$ because ${f_{k}}$ is a decreasing series, and there is a constant $f^{*}$ that results in

(21)

lim_{k \to \infty} f_{k} = f^{*}

3. Let Q be a matrix from the 2^nd derivatives of the $f$ . Then, there exist constants $R$ and $r$ , such that:

(22)

r {‖ z ‖}^{2} \leq z^{T} Qz \leq R {‖ z ‖}^{2}

for all $z \in R^{n}$ , for more details see.^12–14

Theorem 2.

If ${x_{k}}$ is generated using the proposed algorithm. Then we have:

(23)

r {‖ s_{k} ‖}^{2} \leq s_{k}^{T} \tilde{y_{k}} \leq R {‖ s_{k} ‖}^{2} .

and

(24)

‖ \tilde{y_{k}} ‖ \leq (L + R) ‖ s_{k} ‖ .

Proof:

By different gradient definitions $\tilde{y_{k}}$ and combining Equations (10) with (16), we obtain:

(25)

s_{k}^{T} \tilde{y_{k}} = s_{k}^{T} Q (x_{k}) s_{k} = \frac{2}{3} s_{k}^{T} y_{k} + \frac{2}{3} (f_{k} - f_{k + 1}) = 2 (f_{k + 1} - f_{k}) - 2 s_{k}^{T} g_{k} .

Utilizing the mean value theorem and Taylor series, we obtain:

(26)

f_{k + 1} = f_{k} + s_{k}^{T} g_{k} + \frac{1}{2} s_{k}^{T} Q (η_{k}) s_{k}

where

ξ \in (0, 1)

and

η_{k} = x_{k} + ξ (x_{k + 1} - x_{k})

. As such by Eqs. (25) and (26), as follows:

(27)

s_{k}^{T} \tilde{y_{k}} = 2 (s_{k}^{T} g_{k} + \frac{1}{2} s_{k}^{T} Q (η_{k}) s_{k}) - 2 s_{k}^{T} g_{k} = 2 s_{k}^{T} g_{k} + s_{k}^{T} Q (η_{k}) s_{k} - 2 s_{k}^{T} g_{k} = s_{k}^{T} Q (η_{k}) s_{k}

Meeting Assumption 3, it is simple to surmise:

(28)

r {‖ s_{k} ‖}^{2} \leq s_{k}^{T} \tilde{y_{k}} \leq R {‖ s_{k} ‖}^{2}

Then, we obtain different gradient definitions of $\tilde{y_{k}}$ by direct calculations:

(29)

‖ \tilde{y_{k}} ‖ = ‖ \frac{2}{3} y_{k} + \frac{[2 / 3 (f_{k} - f_{k + 1})]}{s_{k}^{T} u_{k}} u_{k} ‖ \leq \frac{2}{3} ‖ y_{k} ‖ + \frac{| [s_{k}^{T} Q (η_{k}) s_{k} - 2 / 3 (s_{k}^{T} y_{k})] |}{‖ δ_{k} ‖ ‖ u_{k} ‖} ‖ u_{k} ‖ \leq \frac{4}{3} ‖ y_{k} ‖ + \frac{| [s_{k}^{T} Q (η_{k}) s_{k}] |}{‖ s_{k} ‖} \leq 4 / 3 L ‖ s_{k} ‖ + R ‖ s_{k} ‖ \leq (4 / 3 L + R) ‖ s_{k} ‖

The proof is finished.

Theorem 3.

If the constants $a_{1} > 0$ and $a_{2} > 0$ exist, then the following inequality holds:

(30)

s_{k}^{T} B_{k} s_{2} \geq a_{2} {‖ s_{k} ‖}^{2}, and ‖ B_{k} s_{k} ‖ \leq a_{1} ‖ s_{k} ‖

for any

k

. The sequence

{x_{k}}

is obtained using the new algorithm, and we obtain:

(31)

lim_{k \to \infty} inf ‖ g_{k} ‖ = 0 .

Proof:

The proof is straightforward, similar to the proof of Theorem 3 in.⁶

In this study, we prove a global convergence theorem for non-convex problems and suggest a cautious updating strategy that is comparable to that mentioned previously. We state a Powell-related lemma for motivational purposes.¹⁵

Lemma 1.

A smooth function f that is limited below can be treated using the BFGS technique if a constant $M > 0$ exists, which makes the inequality:

(32)

{‖ \tilde{y_{k}} ‖}^{2} / s_{k}^{T} \tilde{y_{k}} \leq M

then:

(33)

lim_{k \to \infty} inf ‖ g_{k} ‖ = 0 .

Theorem 4.

If these Assumptions hold, ${x_{k}}$ is generated by the new algorithm. Then Eq. (32) holds.

Proof:

If Eq.(33) fails to hold, then there exists a constant $ε > 0$ , such that:

(34)

‖ g_{k} ‖ \geq ε .

Therefore, a constant $r > 0$ exists, such that:

(35)

r {‖ s_{k} ‖}^{2} \leq s_{k}^{T} \tilde{y_{k}} .

So, combining Eqs. (29) and (35) imply that:

(36)

{‖ \tilde{y_{k}} ‖}^{2} / s_{k}^{T} \tilde{y_{k}} \leq M .

The proof is finished.

4. Numerical experiments

In this section, we present a numerical comparison of QN -techniques and suggest modifications for solving 4^th order nonlinear partial differential equations.

Example 1:

Consider the nonlinear 4^th order has the form;

\begin{matrix} ʯ_{xt} - ʯ_{xxxy} - 2 ʯ_{xx} ʯ_{y} - 4 ʯ_{x} ʯ_{xy} = 0; \\ ʯ (x, y, 0) = \frac{1}{2} {sech}^{2} (\frac{1}{2} (x + y)) and, \\ exact solution ʯ (x, y, z, t) = tanh (\frac{1}{2} (x + y - t)) \end{matrix}

The results of solving the above equation at different times t are presented in Table 1. The neural solution for this equation is shown in Figure 1.

We stopped utilizing the algorithms by employing Himmeblau's law¹⁸:

If $| f (x_{k}) | > 1 0^{- 5},$ then $\frac{| f (x_{k}) - f (x_{k + 1}) |}{| f (x_{k}) |} = 1$ . Otherwise, $| f (x_{k}) - f (x_{k + 1}) | = 1$ . For every problem, if $‖ g_{k} ‖ < ε$ is used, the program is terminated.

Quasi-Newton approaches perform better when an appropriate quasi-Newton equation is employed. The performance of the new update with $u_{k} = g_{k + 1}$ was the best of the three methods, whereas the normal performance of the new update with $u_{k} = y_{k}$ and $u_{k} = g_{k}$ was somewhat better than that of the BFGS technique. As a result, among the QN -procedures for unconstrained problems, the new update with $u_{k} = g_{k + 1}$ is the most well -organized.

Example 2:

Consider the nonlinear 4^th order has the form:

\begin{matrix} ʯ_{tt} - ʯ_{xx} - ʯ_{xxxx} - ʯ_{yy} - ʯ_{zz} - 3 {(ʯ^{2})}_{xx} = 0 \\ ʯ (x, y, z, 0) = \frac{1}{2} {sech}^{2} (\frac{1}{2} (x + y + z)), u_{t} = tanh (\frac{1}{2} (x + y + z)) {sech}^{2} (\frac{1}{2} (x + y + z)) \end{matrix}

Exact solution:

ʯ (x, y, z, t) = \frac{1}{2} {sech}^{2} (\frac{1}{2} (x + y + z - 2 t))

The neural solution for this equation is shown in Figure 2 when z = -0.5. The accuracy for epochs and time is presented in Table 2, and Table 3, illustrates the results of the neural solution of the equation.

Table 1. The results of suggested algorithm for different values of time t.

X = y	ti exact	Suggested update
X = y	ti exact	t = 0.001	t = 0.01	t = 0.05	t = 0.25	t = 0. 5
0	-0.000499999958333338	-0.000048659724380	-0.00499995832713615	-0.025004418506876	-0.124353001771672	-0.244918662401479
0.1	0.0991729368500791	0.099174522493650	0.0947152247011525	0.074859690643595	-0.0249947929685649	-0.148885033624227
0.2	0.196894751347250	0.196894751347288	0.192565398608004	0.173235732159165	0.0748596906873580	-0.0499583749589804
0.3	0.290854977351376	0.290854977351250	0.286730291373398	0.268271182008229	0.173235157834554	0.0499583749579298
0.4	0.379521061607639	0.379521061607816	0.375662661174346	0.358357398344881	0.268271160988048	0.148885033623492
0.5	0.461723842547565	0.461723842547454	0.458175852175461	0.442230453940485	0.358357335349861	0.244918662402002
0.6	0.536693682582613	0.536693686709420	0.533482128457157	0.519021833904887	0.442230290513323	0.336375352939167
0.7	0.604050311415608	0.604050311415511	0.601184473121516	0.588259256403465	0.519021833898177	0.421898609908564
0.8	0.663757149868171	0.663757149868364	0.661232203097477	0.649827607630977	0.588259256398005	0.500520211189160
0.9	0.716054324313046	0.716054380560282	0.713854553039899	0.703905603862037	0.649827419353020	0.571668985813867
1	0.761384088809508	0.761384088809337	0.759486275064505	0.750893283626045	0.703905603936521	0.635140845030389

Figure 1. Illustration the results using new algorithm for different time t.

Figure 2. Solution for z = -1/2.

Table 2. Properties of the proposed algorithm for solving Example 1.

Train Function “Trainbfg”	Performance of train	Epoch	Time	Msereg
[t = 0.001]	4.72e-27	818	0:00:02	1.4903e-11
[t = 0.01]	7.27e-23	404	0:00:00	3.0524e-17
[t = 0.05]	9.34e-24	33	0:00:00	7.6100e-12
[t = 0.25]	2.64e-27	909	0:00:01	8.7302e-16
[t = 0. 5]	1.59e-24	593	0:00:01	5.4723e-12

Table 3. MSE and performance for training, validation, and testing for the solution of Example 2.

	LM	Suggested update BFG	SCG	RP
Time	00:00:39	00:00: 8	00:00:44	00:00:12
Best Epoch	1000	810	1000	1000
MSE	2.61912e-12	6.9328543e-17	2.9424106e-07	5.9553091e-06
Best training perf	2.694601e-12	6.694813e-14	2.21545518e-07	5.9894044e-06
Best validation perf	2.334575e-12	7.2694735e-16	1.996644e-07	5.7156087e-06
Best test perf	2.5514463e-12	7.7070942e-15	2.254638e-07	6.0358983e-06

5. Conclusions

In this study, we constructed improved BFGS quasi-Newton updating formulae by using the proposed robust QN -equation. Second-order information from Hessian’s Hessian objective function Hessian’s is used in this study to develop a novel quasi-Newton equation. Two nonlinear 4^th order example are provided to illustrate the accuracy of the suggested update, The results are consistent with the practical results and conform to the results that the suggested update, is globally convergent.

Data availability

No data were included in this study.

References

1. Hassan MA: Estimate the Effect of Rainwaters in Contaminated Soil by Using Simulink Technique. JPCS. 2018; 1003: 012057. 1-6.
2. Jamil HJ, Al-Noor TH: Hookah Smoking with Health Risk Perception of Different Types of Tobacco. JPCS. 2020; 1664(1): 012127. 1-11. Publisher Full Text
3. Hussein NA: Solitary Wave Solution of Zakharov-Kuznetsov Equation. AIP Conf. Proc. 2022; 2398(1): 060084. 1-6.
4. Ali MH: Design optimal neural network for solving unsteady state confined aquifer problem. Mathematical Modelling of Engineering Problems. 2023; 10(2): 565–571. Publisher Full Text
5. Kareem ZH: Solving Three-Dimensional Groundwater Recharge Based on Decomposition Method. JPCS. 2020; 1530: 012068. 1-8. Publisher Full Text
6. Al-Noor NH, Al-Noor TH: Estimate the rate of contamination in Baghdad soils by using numerical method. JPCS. 2019; 1294(3): 032020. 1-10.
7. Khamas AH: New approach for calculate exponential integral function. Iraqi Journal of Science. 2023; 64(8): 4034–4042.
8. Abood IN: Persons’ Camp Using Interpolation Method. JPCS. 2018; 1003: 012055. 1-9.
9. Enadi MO: New approach for solving three-dimensional space partial differential equation. Baghdad Sci. J. 2019; 16(3): 786–792.
10. Ghazi FF: New Approach for Solving Two- dimensional Spaces. PDE, JPCS. 2020; 1530: 012066. 1-9. Publisher Full Text
11. Hussein NA: Efficient approach for solving high-order (2+1) D-differential equation. AIP Conf. Proc. 2022; 2398: 060018. 1-7.
12. Kareem ZH: Recent modification of decomposition method for solving wave-like equations. Journal of Interdisciplinary Mathematics. 2023; 26(5): 809–820. Publisher Full Text
13. Salih OM: Design neural network based upon decomposition approach for solving reaction diffusion equation. JPCS. 2019; 1234(1): 012104. Publisher Full Text
14. Hussein NA: New Approach for Solving (1+1)-Dimensional Differential Equation. JPCS. 2020; 1530: 012098. 1-10. Publisher Full Text
15. Kareem ZH: Efficient Modification of Decomposition Method for Solving a System of PDEs. Iraqi Journal of Science. 2021; 62(9): 3061–3070.
16. Ali MH, Tawfiq LNM: Novel Neural Network Based on New Modification of BFGS Update Algorithm for Solving Partial Differential Equations. Advances in the Theory of Nonlinear Analysis and its Applications. 2023; 7(4): 76–88.
17. Hussein NA: Efficient Approach for Solving (2+1) D- Differential Equations. Baghdad Sci. J. 2023; 20(1): 166–174.
18. Kareem ZH: New soliton solution of nonlinear (3+1) D-mKd, (3+1)D-KP, and (3+1)D-gKP equation. AIP Conference Proceedings. 2024; 3036(1): 040034. 1-12.
19. Khamas AH: Determine the Effect Hookah Smoking on Health with Different Types of Tobacco by using parallel processing technique. JPCS. 2021; 1818: 012175. 1-10. Publisher Full Text
20. Ghazi FF: Design optimal neural network based on new LM training algorithm for solving 3D-PDEs. An International Journal of Optimization and Control: Theories & Applications. 2024; 14(3): 249–260. Publisher Full Text
21. Hussein NA: Exact solutions for systems of nonlinear (2+1)D-Differential Equations. Iraqi Journal of Science. 2022; 63(10): 4388–4396.
22. Helal MM: Double LA transform and their properties for solving partial differential equations. AIP Conference Proceedings. 2023; 2834(1): 080140. 1-10.
23. Ibrahim AQ: Efficient method for solving fourth-order PDEs. JPCS. 2021; 1818: 012166. 1-9.
24. Kareem ZH: Solving (3+1) D-New Hirota Bilinear Equation Using Tanh Method and New Modification of Extended Tanh Method. Advances in the Theory of Nonlinear Analysis and its Applications. 2023; 7(4): 114–122.
25. Ali MH: Efficient design of neural networks for solving third-order partial differential equations. JPCS. 2020; 1530(1): 012067. Publisher Full Text
26. Hussein NA: Exact soliton solution for systems of nonlinear (2+1)D-DEs. AIP Conference Proceedings. 2023; 2834(1): 080137.
27. Kareem ZH: New modification of decomposition method for solving high-order strongly nonlinear partial differential equations. AIP Conference Proceedings. 2022; 2398: 0600363. 1-9.
28. Mohammed MA, Tawfiq LNM: New accurate method for solving fourth-order two-dimension evolution equations. Journal of Interdisciplinary Mathematics. 2025; 28(3): 1269–1276. Publisher Full Text
29. Hussein NA: New Approach for Solving (2+1)-Dimensional Differential Equation. JPCS. 2021; 1818(1): 012182. Publisher Full Text
30. Ghazi FF: Modeling the Contamination of Soil Adjacent to Mohammed Al-Qassim Highway in Baghdad, Iraqi Journal of. Science. 2020; Vol. 61(10): pp: 2663–2670. (2020). Publisher Full Text
31. Thirthar AA, Shah K, Abdeljawad T: Design an efficient neural network for solving steady state problems. J. Math. Comput. Sci. 2025; 36(1): 84–98. Publisher Full Text
32. Ali MH: Efficient design of neural network based on new BFGS training algorithm for solving ZK-BBM equation. AIP Conference Proceedings. 2024; 3036(1): 040009.
33. Abdul-Jabbar SA: Design Efficient Neural Network for Solving 2D-Nonlinear Wave-Like Equations. AIP Conference Proceedings. 2025; 3264(1): 050055.
34. Ghazi FF: Solving 5th Order nonlinear 4D-PDEs Using Efficient Design of Neural Network. AIP Conference Proceedings. 2025; 3264(1): 050042.

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 16 Jan 2026

Author details Author details

¹ Mathematics, University of Baghdad, Baghdad, Iraq
² Ministry of Education, Directorate General of Education, KARKH II, Baghdad, Iraq

Farah Ghazi
Roles: Data Curation, Resources, Software, Validation

Luma Tawfiq
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Review & Editing

Zainab Kareem
Roles: Methodology, Project Administration, Visualization, Writing – Original Draft Preparation

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (1)

version 1

Published: 16 Jan 2026, 15:71

https://doi.org/10.12688/f1000research.172826.1

Copyright

© 2026 Ghazi F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The author(s) is/are employees of the US Government and therefore domestic copyright protection in USA does not apply to this work. The work may be protected under the copyright laws of other jurisdictions when used in those jurisdictions.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Ghazi F, Tawfiq L and Kareem Z. Update Quasi-Newton Algorithm for Training ANN [version 1; peer review: awaiting peer review]. F1000Research 2026, 15:71 (https://doi.org/10.12688/f1000research.172826.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 16 Jan 2026

Open Peer Review

Reviewer Status

AWAITING PEER REVIEW

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

[1] 1. Hassan MA: Estimate the Effect of Rainwaters in Contaminated Soil by Using Simulink Technique. JPCS. 2018; 1003: 012057. 1-6.

[2] 2. Jamil HJ, Al-Noor TH: Hookah Smoking with Health Risk Perception of Different Types of Tobacco. JPCS. 2020; 1664(1): 012127. 1-11. Publisher Full Text

[3] 3. Hussein NA: Solitary Wave Solution of Zakharov-Kuznetsov Equation. AIP Conf. Proc. 2022; 2398(1): 060084. 1-6.

[4] 4. Ali MH: Design optimal neural network for solving unsteady state confined aquifer problem. Mathematical Modelling of Engineering Problems. 2023; 10(2): 565–571. Publisher Full Text

[5] 5. Kareem ZH: Solving Three-Dimensional Groundwater Recharge Based on Decomposition Method. JPCS. 2020; 1530: 012068. 1-8. Publisher Full Text

[6] 6. Al-Noor NH, Al-Noor TH: Estimate the rate of contamination in Baghdad soils by using numerical method. JPCS. 2019; 1294(3): 032020. 1-10.

[7] 7. Khamas AH: New approach for calculate exponential integral function. Iraqi Journal of Science. 2023; 64(8): 4034–4042.

[8] 8. Abood IN: Persons’ Camp Using Interpolation Method. JPCS. 2018; 1003: 012055. 1-9.

[9] 9. Enadi MO: New approach for solving three-dimensional space partial differential equation. Baghdad Sci. J. 2019; 16(3): 786–792.

[10] 10. Ghazi FF: New Approach for Solving Two- dimensional Spaces. PDE, JPCS. 2020; 1530: 012066. 1-9. Publisher Full Text

[11] 11. Hussein NA: Efficient approach for solving high-order (2+1) D-differential equation. AIP Conf. Proc. 2022; 2398: 060018. 1-7.

[12] 12. Kareem ZH: Recent modification of decomposition method for solving wave-like equations. Journal of Interdisciplinary Mathematics. 2023; 26(5): 809–820. Publisher Full Text

[13] 13. Salih OM: Design neural network based upon decomposition approach for solving reaction diffusion equation. JPCS. 2019; 1234(1): 012104. Publisher Full Text

[14] 14. Hussein NA: New Approach for Solving (1+1)-Dimensional Differential Equation. JPCS. 2020; 1530: 012098. 1-10. Publisher Full Text

[15] 15. Kareem ZH: Efficient Modification of Decomposition Method for Solving a System of PDEs. Iraqi Journal of Science. 2021; 62(9): 3061–3070.

[16] 16. Ali MH, Tawfiq LNM: Novel Neural Network Based on New Modification of BFGS Update Algorithm for Solving Partial Differential Equations. Advances in the Theory of Nonlinear Analysis and its Applications. 2023; 7(4): 76–88.

[17] 17. Hussein NA: Efficient Approach for Solving (2+1) D- Differential Equations. Baghdad Sci. J. 2023; 20(1): 166–174.

[18] 18. Kareem ZH: New soliton solution of nonlinear (3+1) D-mKd, (3+1)D-KP, and (3+1)D-gKP equation. AIP Conference Proceedings. 2024; 3036(1): 040034. 1-12.

[19] 19. Khamas AH: Determine the Effect Hookah Smoking on Health with Different Types of Tobacco by using parallel processing technique. JPCS. 2021; 1818: 012175. 1-10. Publisher Full Text

[20] 20. Ghazi FF: Design optimal neural network based on new LM training algorithm for solving 3D-PDEs. An International Journal of Optimization and Control: Theories & Applications. 2024; 14(3): 249–260. Publisher Full Text

[21] 21. Hussein NA: Exact solutions for systems of nonlinear (2+1)D-Differential Equations. Iraqi Journal of Science. 2022; 63(10): 4388–4396.

[22] 22. Helal MM: Double LA transform and their properties for solving partial differential equations. AIP Conference Proceedings. 2023; 2834(1): 080140. 1-10.

[23] 23. Ibrahim AQ: Efficient method for solving fourth-order PDEs. JPCS. 2021; 1818: 012166. 1-9.

[24] 24. Kareem ZH: Solving (3+1) D-New Hirota Bilinear Equation Using Tanh Method and New Modification of Extended Tanh Method. Advances in the Theory of Nonlinear Analysis and its Applications. 2023; 7(4): 114–122.

[25] 25. Ali MH: Efficient design of neural networks for solving third-order partial differential equations. JPCS. 2020; 1530(1): 012067. Publisher Full Text

[26] 26. Hussein NA: Exact soliton solution for systems of nonlinear (2+1)D-DEs. AIP Conference Proceedings. 2023; 2834(1): 080137.

[27] 27. Kareem ZH: New modification of decomposition method for solving high-order strongly nonlinear partial differential equations. AIP Conference Proceedings. 2022; 2398: 0600363. 1-9.

[28] 28. Mohammed MA, Tawfiq LNM: New accurate method for solving fourth-order two-dimension evolution equations. Journal of Interdisciplinary Mathematics. 2025; 28(3): 1269–1276. Publisher Full Text

[29] 29. Hussein NA: New Approach for Solving (2+1)-Dimensional Differential Equation. JPCS. 2021; 1818(1): 012182. Publisher Full Text

[30] 30. Ghazi FF: Modeling the Contamination of Soil Adjacent to Mohammed Al-Qassim Highway in Baghdad, Iraqi Journal of. Science. 2020; Vol. 61(10): pp: 2663–2670. (2020). Publisher Full Text

[31] 31. Thirthar AA, Shah K, Abdeljawad T: Design an efficient neural network for solving steady state problems. J. Math. Comput. Sci. 2025; 36(1): 84–98. Publisher Full Text

[32] 32. Ali MH: Efficient design of neural network based on new BFGS training algorithm for solving ZK-BBM equation. AIP Conference Proceedings. 2024; 3036(1): 040009.

[33] 33. Abdul-Jabbar SA: Design Efficient Neural Network for Solving 2D-Nonlinear Wave-Like Equations. AIP Conference Proceedings. 2025; 3264(1): 050055.

[34] 34. Ghazi FF: Solving 5th Order nonlinear 4D-PDEs Using Efficient Design of Neural Network. AIP Conference Proceedings. 2025; 3264(1): 050042.

Update Quasi-Newton Algorithm for Training ANN

Abstract

Keywords

1. Introduction

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

2. Derivation of suggested update

(10)

(11)

(12)

(13)

(14)

(15)

(16)

Theorem 1.

Proof.

(17)

(18)

(19)

3. Convergent analysis

(20)

(21)

(22)

Theorem 2.

(23)

(24)

Proof:

(25)

(26)

(27)

(28)

(29)

Theorem 3.

(30)

(31)

Proof:

Lemma 1.

(32)

(33)

Theorem 4.

Proof:

(34)

(35)

(36)

4. Numerical experiments

Example 1:

Example 2:

Table 1. The results of suggested algorithm for different values of time t.

Figure 1. Illustration the results using new algorithm for different time t.

Figure 2. Solution for z = -1/2.

Table 2. Properties of the proposed algorithm for solving Example 1.

Table 3. MSE and performance for training, validation, and testing for the solution of Example 2.

5. Conclusions

Data availability

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated