426. Bilevel hyperparameter optimization for RBF Kernel support vector machines
Invited abstract in session MB-7: Hyperparameter Optimization for Classification, stream Bilevel and multilevel optimization.
Monday, 10:30-12:30Room: B100/5015
Authors (first author is the speaker)
| 1. | Qingna Li
|
| School of Mathematics and Statistics, Beijing Institute of Technology |
Abstract
The problem of tuning the hyperparameters of a Support Vector Machine (SVM) model via cross validation is intuitively a bilevel problem. Methods for solving this problem have been presented in the literature however these papers have addressed only the linear kernel SVM. The Radial Basis Function (RBF) or Gaussian kernel affords SVM models the ability to capture more complex relationships between the variables of our data. This however comes with the drawback of the primal form of the training problem containing the function $\phi: \mathbb{R} \rightarrow \mathbb{R}^\infty$. To avoid this, we consider instead the dual formulation of problem. Therefore, we have to construct robust methods for deriving the primal parameters from the dual parameters. We perform the KKT single level reformulation and apply the Sholtes relaxation to solve the resulting mathematical program with equilibrium constraints (MPEC). The relaxed problem is then solved using fmincon in MATLAB. We also discussed the theoretical property of the resulting MPEC.
Keywords
- Applications of continuous optimization
- Multi-level optimization
- Optimization for learning and data analysis
Status: accepted
Back to the list of papers