520. Line search based stochastic gradient methods for learning applications
Invited abstract in session MB-3: First-order methods in modern optimization (Part I), stream Large scale optimization: methods and algorithms.
Monday, 10:30-12:30Room: B100/4011
Authors (first author is the speaker)
| 1. | Federica Porta
|
| Universita' di Modena e Reggio Emilia |
Abstract
Empirical risk minimization problems arise in a variety of applications, including machine learning and deep learning. Stochastic gradient algorithms are a standard approach to solve these problems due to their low computational cost per iteration and a relatively simple implementation. The aim of this talk is to present stochastic gradient algorithms based on Armiko-like line search to automatically select the learning rate, reducing the need for manual hyperparameter tuning, which can be computationally intensive. The line search can be combined with both acceleration techniques such as inertial term and/or variable metric and dynamical variance reduction procedures. Convergence properties of the resulting algorithms can be proved under the assumptions of both non-convex and convex objective functions. Numerical experiments on classification tasks illustrate the effectiveness and practical benefits of the proposed strategies.
Keywords
- Stochastic optimization
- First-order optimization
Status: accepted
Back to the list of papers