EURO 2024 Copenhagen
Abstract Submission

EURO-Online login

20. A multiobjective continuation method to compute the regularization path of deep neural networks

Invited abstract in session TB-37: Advances in Continuous Multiobjective Optimization, stream Multiobjective Optimization.

Tuesday, 10:30-12:00
Room: 33 (building: 306)

Authors (first author is the speaker)

1. Augustina Chidinma Amakor
Computer Science, Universität Paderborn
2. Sebastian Peitz
Department of Computer Science, University of Paderborn

Abstract

Sparsity is a highly desired feature in deep neural networks (DNNs) since it ensures numerical efficiency, improves the interpretability of models (due to the smaller number of relevant features), and robustness. In machine learning approaches based on linear models, it is well known that there exists a connecting path between the sparsest solution in terms of the L1 norm (i.e., zero weights) and the non-regularized solution, which is called the regularization path. Very recently, there was a first attempt to extend the concept of regularization paths to DNNs by means of treating the empirical loss and sparsity (L1 norm) as two conflicting criteria and solving the resulting multiobjective optimization problem. However, due to the non-smoothness of the L1 norm and the high number of parameters, this approach is not very efficient from a computational perspective. To overcome this limitation, we present an algorithm that allows for the approximation of the entire Pareto front for the above-mentioned objectives in a very efficient manner. We present numerical examples using both deterministic and stochastic gradients. We furthermore demonstrate that knowledge of the regularization path allows for a well-generalizing network parametrization.

Keywords

Status: accepted

Back to the list of papers