24. SDP Relaxations for Training ReLU Activation Neural Networks
Invited abstract in session TB-6: Advances in Semi-Definite Programming, stream Challenges in nonlinear programming.
Thursday, 10:05 - 11:20Room: M:H
Authors (first author is the speaker)
| 1. | Karthik Prakhya
|
| Umeå University | |
| 2. | TOLGA BIRDAL
|
| Computing, Imperial College London | |
| 3. | Alp Yurtsever
|
| Umeå University |
Abstract
Solving non-convex optimization problems is crucial for training machine learning models, including neural networks. However, non-convexity often leads to less reliable and less robust neural networks with unclear inner workings. While convex formulations have been used for verifying neural network robustness, their application to training neural networks remains relatively unexplored. In this work, we propose a semidefinite programming relaxation for training two-layer ReLU networks in a lifted space, which can be solved in polynomial time. Numerical experiments demonstrate that this SDP formulation provides reasonably tight lower bounds on the training objective across various prediction and classification tasks.
Keywords
- SS - Conic Optimization and Applications
- SS - Semidefinite Optimization
- Optimization for learning and data analysis
Status: accepted
Back to the list of papers