Can I read Deep Exploration with PAC-Bayes on EtoBox?

Deep Exploration with PAC-Bayes by Tasdighi, Bahareh; Haussmann, Manuel; Werge, Nicklas; Wu, Yi-Shan; Kandemir, Melih is a scholarly article available to read on EtoBox.

What is Deep Exploration with PAC-Bayes about?

Reinforcement learning for continuous control under delayed rewards is an under-explored problem despite its significance in real life. Many complex skills build on intermediate ones as prerequisites. For instance, a humanoid locomotor has to learn how to stand before it can learn to walk. To cope with delayed reward, a reinforcement learning agent has to perform deep exploration. However, existing deep exploration methods are designed for small discrete action spaces, and their successful generalization to state-of-the-art continuous control remains unproven. We address the deep exploration problem for the first time from a PAC-Bayesian perspective in the context of actor-critic learning. To do this, we quantify the error of the Bellman operator through a PAC-Bayes bound, where a bootstrapped ensemble of critic networks represents the posterior distribution, and their targets serve as a data-informed function-space prior. We derive an objective function from this bound and use it to train the critic ensemble. Each critic trains an individual soft actor network, implemented as a shared trunk and critic-specific heads. The agent performs deep exploration by acting epsilon-greedily o

Author

Tasdighi, Bahareh; Haussmann, Manuel; Werge, Nicklas; Wu, Yi-Shan; Kandemir, Melih

Published

2024

Language

EN