EURO 2024 Copenhagen
Abstract Submission

EURO-Online login

1279. Measurized Markov Decision Processes

Invited abstract in session TD-33: Optimal control theory, stream Optimal Control Theory and Applications.

Tuesday, 14:30-16:00
Room: 42 (building: 303A)

Authors (first author is the speaker)

1. Alba V Olivares-Nadal
UNSW Business School

Abstract

In this talk, we introduce a framework that facilitates the analysis of discounted infinite horizon Markov Decision Processes (MDPs) by visualizing them as deterministic processes where the states are probability measures on the original state space and the actions are stochastic kernels on the original action space. More specifically, we provide a simple general algebraic approach to lifting any MDP to this space of measures; we call this to measurize the original stochastic MDP. We show that measurized MDPs are in fact a generalization of stochastic MDPs, thus the measurized framework can be deployed without loss of fidelity. Lifting an MDP can be convenient because the measurized framework enables constraints and value function approximations that are not easily available from the standard MDP setting. For instance, one could add restrictions or build approximations based on moments, quantiles, risk measures...etc. In addition, measurized MDPs are deterministic and are particularly beneficial when managing large populations. A reason is that high-dimensional weakly coupled MDPs can be reduced into a unidimensional MDP in the space of distributions when the state-components are independent and identically distributed. This implies that solving the measurized problem only requires aggregated information, eliminating the impractical assumption of constantly having detailed and updated information for each component in high-dimensional contexts.

Keywords

Status: accepted

Back to the list of papers