508. Communication-Efficient Algorithms for Federated Learning and Weakly Coupled Games
Invited abstract in session TB-5: Randomized Optimization algorithms I, stream Optimization for machine learning.
Tuesday, 10:30-12:30Room: B100/4013
Authors (first author is the speaker)
| 1. | Sebastian Stich
|
| CISPA Helmholtz Center for Information Security | |
| 2. | Ali Zindari
|
| CISPA Helmholtz Center | |
| 3. | Parham Yazdkhasti
|
| CISPA Helmholtz Center | |
| 4. | Anton Rodomanov
|
| CISPA Helmholtz Center for Information Security | |
| 5. | Tatjana Chavdarova
|
| EECS, UC Berkeley |
Abstract
Communication-efficient algorithms are crucial for distributed and federated learning, where reducing communication overhead can significantly enhance performance. Over the past decade, many such algorithms have been developed and refined. However, these algorithms often only reduce communication in certain regimes, and in the worst-case scenarios, they may not reduce the communication required to reach a predefined level of accuracy.
In this talk, we will explore this question and review the speedup properties of several algorithms. We will focus on those that can provably reduce the total communication cost of training. One such method is \emph{Decoupled SGDA}, a novel adaptation we developed for distributed games. Our findings show that, in the context of \emph{Weakly Coupled Games}, \emph{Decoupled SGDA} can achieve communication speed-up, making it a promising approach for resource-constrained environments.
Keywords
- Optimization for learning and data analysis
- Distributed optimization
Status: accepted
Back to the list of papers