Can I read Disentangling Causal Effects for Hierarchical Reinforcement Learning on EtoBox?

Disentangling Causal Effects for Hierarchical Reinforcement Learning by Corcoll, Oriol; Vicente, Raul is a scholarly article available to read on EtoBox.

What is Disentangling Causal Effects for Hierarchical Reinforcement Learning about?

Exploration and credit assignment under sparse rewards are still challenging problems. We argue that these challenges arise in part due to the intrinsic rigidity of operating at the level of actions. Actions can precisely define how to perform an activity but are ill-suited to describe what activity to perform. Instead, causal effects are inherently composable and temporally abstract, making them ideal for descriptive tasks. By leveraging a hierarchy of causal effects, this study aims to expedite the learning of task-specific behavior and aid exploration. Borrowing counterfactual and normality measures from causal literature, we disentangle controllable effects from effects caused by other dynamics of the environment. We propose CEHRL, a hierarchical method that models the distribution of controllable effects using a Variational Autoencoder. This distribution is used by a high-level policy to 1) explore the environment via random effect exploration so that novel effects are continuously discovered and learned, and to 2) learn task-specific behavior by prioritizing the effects that maximize a given reward function. In comparison to exploring with random actions, experimental results

Author

Corcoll, Oriol; Vicente, Raul

Published

2020

Language

EN