158. Optimization dynamics of equivariant neural networks
Invited abstract in session WC-3: Optimization in neural architectures I, stream Optimization in neural architectures: convergence and solution characterization.
Wednesday, 10:05 - 11:20Room: M:J
Authors (first author is the speaker)
| 1. | Axel Flinth
|
| Mathematics and Mathematical Statistics, Umeå University |
Abstract
A symmetry in a learning task is an inductive bias, that one should take advantage of. This is the main tenet behind the development of so called geometric deep learning, which objective is to develop and analyze methods for capturing such symmetries. There are frameworks available which allow oneself to design network architectures manifestly equivariant to almost any given group symmetry. A different approach is to simply train a standard architecture on symmetric (augmented) data. Despite the natural relations between the two strategies, systematic comparisons are far and few.
In this talk, we will present initial some results about the relation of the training dynamics of the two approaches. We will discuss conditions which guarantee that the critical points of both training dynamics coincide. We will see that in these cases, the critical points may still have different stability properties for the two strategies.
Keywords
- Optimization for learning and data analysis
Status: accepted
Back to the list of papers