203. Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions
Invited abstract in session MC-2: Matrix factorization, stream Nonsmooth and nonconvex optimization.
Monday, 14:00-16:00Room: B100/7011
Authors (first author is the speaker)
| 1. | Atharva Awari
|
| Mathematics and Operations Research, University of Mons, Belgium | |
| 2. | Nicolas Gillis
|
| Mathematics and Operational Research, Université de Mons | |
| 3. | Arnaud Vandaele
|
| Mathematics and Operations Research, University of Mons |
Abstract
Low-rank matrix approximations are fundamental in data analysis, machine learning, and signal processing. Traditional approaches, such as the singular value decomposition and nonnegative matrix factorization (NMF), assume a linear relationship between the observed data matrix X and its low-rank factors W and H. However, many real-world datasets exhibit nonlinear structures that cannot be effectively captured using standard linear models. This has led to growing interest in nonlinear matrix decompositions (NMDs), where the goal is to find low-rank factors W and H such that X≈f(WH), where f is an element-wise nonlinear function . Despite the increasing relevance of NMDs, existing algorithms lack the flexibility to handle a wide range of nonlinear functions commonly used in practice. Examples include the ReLU function with f(x)=max(0,x)) useful in the approximation of sparse datasets, the component-wise square with f(x)=x^2 useful in representation of probabilistic circuits, and Min-Max models where the data lies in a certain interval (a,b) with f(x)=min(b,max(a,x)). To bridge this gap, we propose an Alternating Direction Method of Multipliers (ADMM) framework tailored for NMDs. Our method efficiently handles diverse nonlinear models while accommodating different loss functions, including least squares, ℓ1-norm, and the Kullback-Leibler divergence.Additionally, our approach is easily adaptable to other nonlinear functions and loss functions, ensuring broad applicability
Keywords
- Linear and nonlinear optimization
- Optimization for learning and data analysis
- Non-smooth optimization
Status: accepted
Back to the list of papers