222. Non-monotone stochastic line search without overhead for training neural networks
Invited abstract in session WB-1: Advances in stochastic and non-euclidean first order methods, stream Zeroth and first-order optimization methods.
Wednesday, 10:30-12:30Room: B100/1001
Authors (first author is the speaker)
| 1. | Andrea Cristofari
|
| Department of Civil Engineering and Computer Science Engineering, University of Rome "Tor Vergata" | |
| 2. | Leonardo Galli
|
| Mathematics, LMU Munich | |
| 3. | Stefano Lucidi
|
| Department of Computer, Control, and Management Science, University of Rome "La Sapienza" |
Abstract
In this work, we investigate the use of a new non-monotone strategy for stochastic gradient. In order to reduce the number of stepsize computations in the non-monotone line search, appropriate tests are used during the iterations to check if the Polyak stepsize can be accepted independently on the objective decrease. A preliminary theoretical analysis is illustrated and numerical results are given for the training of artificial neural networks on different datasets.
Keywords
- Large-scale optimization
Status: accepted
Back to the list of papers