332. Numerical design of optimized first-order methods
Invited abstract in session TB-2: Solver-based optimization algorithms, stream Conic optimization: theory, algorithms and applications.
Thursday, 10:05 - 11:20Room: M:O
Authors (first author is the speaker)
| 1. | Yassine Kamri
|
| INMA, UCLouvain | |
| 2. | Julien Hendrickx
|
| ICTEAM | |
| 3. | François Glineur
|
| ICTEAM/INMA & CORE, Université catholique de Louvain (UCLouvain) |
Abstract
Large-scale optimization is central to many engineering applications such as machine learning, signal processing, and control. First-order optimization methods are a popular choice for solving problems due to their performance and low computational cost. However, in many applications such as machine learning, their performances are highly dependent on the choice of step sizes. Tuning step sizes of first-order methods is a notoriously hard nonconvex problem. In this talk, we present several approaches to optimize the step sizes of first-order methods, relying on recent developments in Performance Estimation Problems. We compare our results to existing optimized step sizes and provide new results, namely numerically optimized step sizes for cyclic coordinate descent and inexact gradient descent, leading to accelerated convergence rate with and without momentum.
Keywords
- Analysis and engineering of optimization algorithms
- SS - Semidefinite Optimization
- Complexity and efficiency of optimization algorithms
Status: accepted
Back to the list of papers