1485. Learning from data via overparameterization
Invited abstract in session TB-35: Optimization for machine learning and inverse problems, stream Continuous and mixed-integer nonlinear programming: theory and algorithms.
Tuesday, 10:30-12:00Room: Michael Sadler LG15
Authors (first author is the speaker)
| 1. | Cesare Molinari
|
| Università di Genova | |
| 2. | Silvia Villa
|
| Department of Mathematics, MaLGa, università di Genova | |
| 3. | Lorenzo Rosasco
|
| DIBRIS, Universita' di Genova | |
| 4. | Cristian Vega
|
| Dima, University of Genoa | |
| 5. | Hippolyte Labarrière
|
| Università di Genova |
Abstract
Solving data driven problems requires defining complex models and fitting them on data, neural networks being a motivating example. The fitting procedure can be seen as an optimization problem which is often non convex, and hence optimization guarantees hard to derive. An opportunity is provided by viewing the model of interest as a redundant re-parameterization - an overparametrization - of some simpler model for which optimization results are easier to achieve. In this talk, after formalizing the above idea, we review some recent results and derive new ones. In particular, we consider the gradient flow of some classes of linear overparamtetrization and show they correspond to suitable mirror flow on the original parameters. Our main contribution relates to the study of the latter, for which we establish well posed-ness and convergence. The results yields insight on the role of overparametrization for implicit regularization and constrained optimization.
Keywords
- Machine Learning
- Algorithms
- Continuous Optimization
Status: accepted
Back to the list of papers