228. Hyperparameter optimization for kernel-regularized system identification
Invited abstract in session FB-5: Recent advances in bilevel optimization II, stream Bilevel optimization: strategies for complex decision-making.
Friday, 10:05 - 11:20Room: M:N
Authors (first author is the speaker)
| 1. | Lujing Chen
|
| Department of Applied Mathematics and Computer Science, Technical University of Denmark | |
| 2. | Martin Skovgaard Andersen
|
| Department of Applied Mathematics and Computer Science, Technical University of Denmark | |
| 3. | Tianshi Chen
|
| The Chinese University of Hong Kong, Shenzhen |
Abstract
System identification is an example of an inverse problem in which one seeks to estimate a mathematical model of a system based on knowledge of its input and output. In the past decade, significant advances have been made within kernel-regularized methods where prior knowledge is included into the estimation problem in the form of a kernel function with small a number of hyperparameters. The problem of finding a suitable set of hyperparameters can be formulated as an optimization problem. This is generally a difficult nonconvex problem with an objective function that is expensive to evaluate when dealing with complex models and large data sets.
To address the computational challenges associated with the hyperparameter optimization problem, we explore the use of cheap, approximate function evaluations combined with Bayesian optimization. Specifically, we employ Krylov subspace methods and stochastic trace estimation with randomized preconditioning to compute an approximate function value. Our methodology yields promising results, as demonstrated by numerical experiments.
Keywords
- Derivative-free optimization
- Optimization for learning and data analysis
Status: accepted
Back to the list of papers