Application of spectral conjugate gradient methods for solving unconstrained optimization problems
Conjugate gradient (CG) methods are among the most efficient numerical methods for solving unconstrained optimization problems. This is due to their simplicty and less computational cost in solving large-scale nonlinear problems. In this paper, we proposed some spectral CG methods using the classical CG search direction. The proposed methods are applied to real-life problems in regression analysis. Their convergence proof was establised under exact line search. Numerical results has shown that the proposed methods are efficient and promising.
[1] Andrei, N. (2008). An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10, 147-161.
[2] Barzilai, J. & Borwein, J.M. (1988). Two-point step size gradient methods, IMA J Numer Anal. 8, 141- 148.
[3] Birgin, E.G. & Martinez, J. M. (2011). A spectral conjugate gradient method for unconstrained optimization, Applied Mathematics and Optimization, 43 (2), 117-128.
[4] Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono, A.M. & Rivaie, M. (2018a). Secant free condition of a spectral PRP conjugate gradient method. International Journal of Engineering & Technology,7 (3.28), 325 – 328.
[5] Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono,A.M. & Rivaie, M. (2018b). Modification on spectral conjugate gradient method for unconstrained optimization. International Journal of Engineering & Technology, 7 (3.28), 307 – 311.
[6] Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & B.Y. Rabi’u. (2018d). Secant free condition of a spectral WYL and its global convergence properties. Far East Journal of Mathematical science, 12, 1889 – 1902.
[7] Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & J. Sabi’u. (2018e). A recent modification on Dai-Liao conjugate gradient method for solving symmetric nonlinear equations. Far East Journal of Mathematical science, 12, 1961 – 1974.
[8] Zull, N. Rivaie, M. Mamat, M. Salleh, Z. & Amani, Z. (2015). Global convergence of a Spectral conjugate gradient by using strong Wolfe line search, Applied Mathematical Sciences, 9(63), 3105- 3117.
[9] Hu, C. & Wan, Z. (2013). An Extended Spectral Conjugate Gradient Method for unconstrained optimization problems, British Journal of Mathematics & Computer Science, 3, 86-98.
[10] X. Wu. (2015). A new spectral Polak- Ribière -Polak conjugate gradient method, ScienceAsia. 41, 345- 349.
[11] Sulaiman, I. M., Sukono, Sudradjat, S., and Mamat, M. (2019). New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search. IOP Conference Series: Materials Science and Engineering, 621(012021), 1- 7.
[12] Mathematical Sciences Issue 65-68, 3307-3319.
[13] W. W. Hager & H. Zhang. (2006). A survey of nonlinear conjugate gradient methods, Pacific Journal of Optimization, 2 (1), 35-58.
[14] Du, X. & Liu, J. (2011). Global convergence of a spectral HS conjugate gradient method, Procedia Engineering. 15, 1487 – 1492.
[15] Raydan, M. (1997). The Barzilai and J.M. Borwein gradient methods for the large scale unconstrained minimization in extreme problems, SIAM Journal on Optimization, 7 (1), 26-33.
[16] Batu, T., Dasgupta, S., Kumar, R., & Rubinfeld, R. (2005). The complexity of approximating the entropy. SIAM Journal on Computing, 35(1): 132– 150.
[17] Motulsky, H., & Christopoulos, A. (2004). Fitting models to biological data using linear and nonlinear regression. Oxford University Press, New York.
[18] Shirin, S., Mahmudul. H., & Laek, S.A. (2015). Age- Structured Population Projection of Bangladesh by Using a Partial Differential Model with Quadratic Polynomial Curve Fitting. Journal of Applied Sciences, 5: 542-551.
[19] Zoutendijk, G. (1970). Nonlinear programming, computational methods, in J Abadie (ED), Integer and Nonlinear Programming, North-Holland, Amsterdam, 37-86.
[20] Yakubu, U.A. (2019). Enhancing spectral conjugate gradient method for solving unconstrained optimization problems. PhD Thesis. University Sultan Zainal Abidin Kuala Terengganu, Malaysia.
[21] Yakubu, U.A. Mamat, M. Mohamad, A.M. Puspa, L.G. & Rivaie, M. (2018c). Secant free condition of a spectral Hestenes-Stiefel conjugate gradient method and its sufficient descent properties. International Journal of Engineering & Technology, 7 (3.28), 312 – 315.
[22] K.E. Hillstrom. (1977). A simulation test approach to the evaluation of the nonlinear optimization algorithm. Journal ACM Trans. Mathematics Software. 3 (4), 305-315.
[23] Dolan, E. & More, J.J. (2002). Benchmarking optimization software with performance profile, Math. Prog. Vol. 91, 201-213.
[24] Z. Wei, S. Yao, and L. Liu. (2006). The convergence properties of some new conjugate gradient methods, Applied Mathematics and Computation, Vol. 183(2), pp. 1341–1350.
[25] E. Polak & G. Ribiere. (1969). Note sur la convergence de directions conjug´ees, Rev. Francaise Informat Recherche Opertionelle, 3e ann´ee, Vol 16, pp. 35–43.
[26] Sulaiman, I. M. Mamat, M. Abashar, A. Rivaie M.(2015). The global convergence properties of an improved conjugate gradient method Applied Mathematical Science, 9(38), pp. 1857-1868.
[27] Powell, M.J.D. (1977). Restart procedures for the conjugate gradient method, Math. Program. 12, 241-254.
[28] Sulaiman, I. M., Mamat, M., Abashar, A., Zabidin, S. (2015): A Modified Nonlinear Conjugate Gradient Method for Unconstrained Optimization, Applied Mathematical Sciences 9(54), pp. 2671- 2682.
[29] Aini, N. Rivaie, M. Mamat, M. and Sulaiman, I. M.(2019). A Hybrid of Quasi-Newton Method with CG Method for Unconstrained Optimization. J. Phys.: Conf. Ser. 1366(012079), 1-10.