205. Coordinate Descent Algorithm for Nonlinear Matrix Decompositions with the ReLU function
Invited abstract in session FC-4: Large-scale optimization III, stream Large-scale optimization.
Friday, 11:25 - 12:40Room: M:M
Authors (first author is the speaker)
| 1. | Atharva Awari
|
| Mathematics and Operations Research, University of Mons, Belgium |
Abstract
Nonlinear Matrix Decompositions (NMD) solve the following problem: Given a matrix X, find low-rank factors W and H such that X is approximated by f(WH), where "f" is an element-wise nonlinear function. In this paper, we focus on the case when "f" is the rectified linear unit (ReLU) activation, that is, the function which maps all negative entries to zero and keeps positive entries the same. This is referred to as ReLU-NMD.
All state-of-the-art algorithms for ReLU-NMD have been designed to solve a reformulation of ReLU-NMD. It turns out that this reformulation leads to a non-equivalent problem, and hence to suboptimal solutions.
In this paper, we propose a coordinate-descent (CD) algorithm designed to solve ReLU-NMD directly. This allows us to compute more accurate solutions, with smaller error. This is illustrated on synthetic and real-world datasets.
Keywords
- Convex and non-smooth optimization
- Optimization for learning and data analysis
- Linear and nonlinear optimization
Status: accepted
Back to the list of papers