590. Evaluation of Multi-agent algorithms in Multi-objective Environments with Weighted Sum Method
Invited abstract in session WC-9: Generalized convexity and monotonicity 4, stream Generalized convexity and monotonicity.
Wednesday, 14:00-16:00Room: B100/8013
Authors (first author is the speaker)
| 1. | Carlos Ignacio Hernández Castellanos
|
| Applied Mathematics and Systems Research Institute, National Autonomous University of Mexico | |
| 2. | José Olivas Díaz
|
| National Autonomous University of Mexico |
Abstract
In many complex problems, collaboration among multiple agents is essential to achieve a common goal. This structure is commonly known as collaborative multi-agent systems. The rise of deep reinforcement learning (DRL) has allowed agents to go further in the environment where they can learn. However, while much of the current MARL research focuses on single-objective tasks, many real-world collaborative multi-agent systems (MAS) problems involve conflicting objectives. This gap drives our study of stochastic multi-objective optimization on MARL (MOMARL) configurations.
Recent literature on Deep Multi-Task Learning (MTL) has defended using weighted sum over more complex multi-task optimizers in supervised and reinforcement learning domains. On the other hand, other works advocate using specialized multi-objective optimization algorithms.
Our study extends this discussion to the multi-agent domain by empirically evaluating nine state-of-the-art MARL algorithms across three environments. Note that these algorithms can be seen as a special case of gradient-based stochastic optimization. This work aims to provide insights into the success of weighted sum on MTL and DRL. For instance, our empirical results show that the Pareto fronts we obtained are highly convex. Notably, extreme weights failed to produce efficient solutions in most instances. Future work will focus on providing a theoretical analysis of these observations and developing effective methods for MOMARL problems.
Keywords
- Optimization for learning and data analysis
- AI based optimization methods
- Multi-objective optimization
Status: accepted
Back to the list of papers