On the Preconditioned Delayed Weighted Gradient Method

R. Aleixo, H. Lara Urdaneta

Abstract


In this article a preconditioned version of the Delayed Weighted Gradient Method (DWGM) is presented and analyzed.  In addition to the convergence, some nice properties as  the A- orthogonality of the current transformed gradient with all the previous gradient vectors as well as finite convergence are demonstrated. Numerical experimentation is also offered, exposing the benefits of preconditioning.

Keywords


Gradient methods, Convex quadratic optimization, Krylov subspace methods, Preconditioning

Full Text:

PDF

References


E. Birgin, I. Chambouleyron, and J. M. Martínez, Estimation of the optical constants and the thickness of thin films using unconstrained optimization, Computational Physics, no. 151, pp. 862-880, 1999.

T. Serani, G. Zanghirati, and L. Zanni, Gradient projection methods for large quadratic programs and applications in training support vector machines, Optimization Methods and Software, no. 20, pp. 353-378, 2005.

M. A. Figueiredo, R. Nowak, and S. Wright, Projection for sparse reconstruction: application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process, no. 1, pp. 586-597, 2007.

A. Cauchy, Méthode générale pour la résolution des systemes d'équations simultanées, Comptes Rendus Sci. Paris, vol. 25, pp. 536-538, 1847.

H. Akaike, On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Annals of the Institute of Statistical Mathematics, vol. 11, pp. 1-16, 1959.

R. D. Asmundis, D. di Serafino, W. W. Hager, G. Toraldo, and H. Zhang,

An eficient gradient method using the Yuan steplength, Computational Optimization and Applications, vol. 59, pp. 541-563, 2014.

J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA Journal of Numerical Analysis, vol. 8, no. 1, pp. 141-148, 1988.

D. di Serano, V. Ruggiero, G. Toraldo, and L. Zanni, On the steplength selection in gradient methods for unconstrained optimization, Applied Mathematics and Computation, vol. 318, pp. 176-195, 2018.

Y.-H. Dai and L.-Z. Liao, R-linear convergence of the Barzilai and Borwein gradient method, IMA Journal of Numerical Analysis, vol. 22, no. 1, pp. 1-10, 2002.

A. Friedlander, J. M. Martínez, B. Molina, and M. Raydan, Gradient method with retards and generalizations, SIAM Journal on Numerical Analysis, vol. 36, no. 1, pp. 275-289, 1999.

Y. X. Yuan, A new stepsize for the steepest decent method, Journal of Computational Mathematics, vol. 24, no. 2, pp. 149-156, 2006.

Y.-H. Dai, Y. Huang, and X.-W. Liu, A family of spectral gradient methods for optimization, Computational Optimization and Applications, vol. 74, pp. 43-65, 2019.

H. F. Oviedo-Leon, A delayed weighted gradient method for strictly convex quadratic minimization, Computational Optimization and Applications, vol. 74, pp. 729-746, 2019.

R. Andreani and M. Raydan, Properties of the delayed weighted gradient method, Computational Optimization and Applications, vol. 78, pp. 167-180, 2021.

G. E. Forsythe and T. S. Motzkin, Asymptotic properties of the optimum gradient method, preliminary report, Bull. Amer. Math. Soc, vol. 57, pp. 304-305, 1951.

B. V. Shah, R. J. Buehler, and O. Kempthorne, Some algorithms for minimizing a function of several variables, Journal of the Society for Industrial and Applied Mathematics, vol. 12, no. 1, pp. 74-92, 1964.

H. Sorenson, Comparison of some conjugate direction procedures for function minimization, Journal of the Franklin Institute, vol. 288, no. 6, pp. 421-441, 1969.

J.-L. Lamotte, B. Molina, and M. Raydan, Smooth and adaptive gradient method with retards, Mathematical and Computer Modelling, vol. 36, pp. 1161-1168, December 2002.

D. Bertaccini and F. Durastante, Iterative Methods and Preconditioning for Large and Sparse Linear Systems with Applications. New York: Chapman and Hall/CRC, 2018.

E. G. Birgin, J. M. Martínez, and M. Raydan, Spectral projected gradient methods: review and perspectives, Journal of Statistical Software, vol. 60, no. 3, pp. 1-21, 2014.

R. D. Asmundis, D. di Serafino, F. Riccio, and G. Toraldo, On spectral properties of steepest descent methods, IMA Journal of Numerical Analysis, vol. 33, no. 4, pp. 1416-1435, 2013.

Y. Huang, Y.-H. Dai, X.-W. Liu, and H. Zhang, Gradient methods exploiting spectral properties, Optimization Methods and Software, vol. 35, no. 4, pp. 681-705, 2020.

G. H. Golub and C. F. Van Loan, Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, Johns Hopkins University Press, 4th ed., 2012.

D. S. Bernstein, Matrix Mathematics: Theory, Facts, and Formulas. Princeton University Press, second ed., 2011.

C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations. Society for Industrial and Applied Mathematics, 1995.[26] W. Ford, Numerical Linear Algebra with Applications: Using MATLAB. New York: Elsevier Inc, 2015.

W. Ford, Numerical Linear Algebra with Applications: Using MATLAB. New York: Elsevier Inc, 2015.




DOI: https://doi.org/10.5540/tcam.2023.024.03.00437

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


Trends in Computational and Applied Mathematics

A publication of the Brazilian Society of Applied and Computational Mathematics (SBMAC)

 

Indexed in:

                       

         

 

Desenvolvido por:

Logomarca da Lepidus Tecnologia