Can I read Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction on EtoBox?

Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction by De Santi, Riccardo; Joseph, Federico Arangath; Liniger, Noah; Mutti, Mirco; Krause, Andreas is a scholarly article available to read on EtoBox.

What is Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction about?

How can a scientist use a Reinforcement Learning (RL) algorithm to design experiments over a dynamical system's state space? In the case of finite and Markovian systems, an area called Active Exploration (AE) relaxes the optimization problem of experiments design into Convex RL, a generalization of RL admitting a wider notion of reward. Unfortunately, this framework is currently not scalable and the potential of AE is hindered by the vastness of experiment spaces typical of scientific discovery applications. However, these spaces are often endowed with natural geometries, e.g., permutation invariance in molecular design, that an agent could leverage to improve the statistical and computational efficiency of AE. To achieve this, we bridge AE and MDP homomorphisms, which offer a way to exploit known geometric structures via abstraction. Towards this goal, we make two fundamental contributions: we extend MDP homomorphisms formalism to Convex RL, and we present, to the best of our knowledge, the first analysis that formally captures the benefit of abstraction via homomorphisms on sample efficiency. Ultimately, we propose the Geometric Active Exploration (GAE) algorithm, which we analys

Author

De Santi, Riccardo; Joseph, Federico Arangath; Liniger, Noah; Mutti, Mirco; Krause, Andreas

Published

2024

Language

EN