Generalized Dai-Yuan conjugate gradient algorithm for training multi-layer feed-forward neural networks
Article Sidebar
Main Article Content
Abstract
In this paper, we will present different type of CG algorithms depending on Peary conjugacy condition. The new conjugate gradient training (GDY) algorithm using to train MFNNs and prove it's descent property and global convergence for it and then we tested the behavior of this algorithm in the training of artificial neural networks and compared it with known algorithms in this field through two types of issues.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Tikrit Journal of Pure Science is licensed under the Creative Commons Attribution 4.0 International License, which allows users to copy, create extracts, abstracts, and new works from the article, alter and revise the article, and make commercial use of the article (including reuse and/or resale of the article by commercial entities), provided the user gives appropriate credit (with a link to the formal publication through the relevant DOI), provides a link to the license, indicates if changes were made, and the licensor is not represented as endorsing the use made of the work. The authors hold the copyright for their published work on the Tikrit J. Pure Sci. website, while Tikrit J. Pure Sci. is responsible for appreciate citation of their work, which is released under CC-BY-4.0, enabling the unrestricted use, distribution, and reproduction of an article in any medium, provided that the original work is properly cited.
References
[1] Bishop, C. and Bishop, C. M. (1995). Neural
networks for pattern recognition. Oxford university
press.
[2] Rumelhart, D. E.; Hinton, G. E., and Williams, R.
J. (1985). Learning internal representations by error
propagation (No. ICS-8506). California Univ San
Diego La Jolla Inst for Cognitive Science.
[3] Hertz, J.; Krogh, A., and Palmer, R. G.
(1991). Introduction to the theory of neural
computation. Addison-Wesley/Addison Wesley
Longman.
[4] Nawi, N. M., Ransing, M. R., & Ransing, R. S.
(2006). An Improved Learning Algorithm based on
the Conjugate Gradient Method for Back Propagation
Neural Networks. Proc of World Academy of Science,
Eng, and Technology, 14.
[5] Beigi, H. S. and Li, C. J. (1993). Learning
algorithms for neural networks based on quasi-
Newton methods with self-scaling. Journal of
dynamic systems, measurement, and control, 115(1):
38-43.
[6] Fletcher, R., and Reeves, C. M. (1964). Function
minimization by conjugate gradients. The computer
journal, 7(2): 149-154.
[7] Poliak, B.T. (1969) .The Conjugate Gradient
Method in Extreme Problems, URSS Comp. Math.
Math. phys., 9(4): 94-112.
[8] Dai, Y. H., & Yuan, Y. (1999). A nonlinear
conjugate gradient method with a strong global
convergence property. SIAM Journal on optimization, 10 (1): 177-182. [9] Hestenes, M. R., and Stiefel, E. (1952). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6): 409-436.
[10] Perry, A. (1978). A modified conjugate gradient algorithm, Operation Research, 26(6): 1073-1078. [11] Dai, Y. H. and Liao, L. Z. (2001). New conjugacy conditions and related nonlinear conjugate gradient methods. Applied Mathematics and Optimization, 43(1): 87-101.
[12] Nguyen, D., and Widrow, B. (1990, June). Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In Neural Networks, 1990., 1990 IJCNN International Joint Conference on (pp. 21-26). IEEE.