Can I read Model-Based Reinforcement Learning with Value-Targeted Regression on EtoBox?

Model-Based Reinforcement Learning with Value-Targeted Regression by Ayoub, Alex; Jia, Zeyu; Szepesvari, Csaba; Wang, Mengdi; Yang, Lin F. is a scholarly article available to read on EtoBox.

What is Model-Based Reinforcement Learning with Value-Targeted Regression about?

This paper studies model-based reinforcement learning (RL) for regret minimization. We focus on finite-horizon episodic RL where the transition model $P$ belongs to a known family of models $\mathcal{P}$, a special case of which is when models in $\mathcal{P}$ take the form of linear mixtures: $P_{\theta} = \sum_{i=1}^{d} \theta_{i}P_{i}$. We propose a model based RL algorithm that is based on optimism principle: In each episode, the set of models that are `consistent' with the data collected is constructed. The criterion of consistency is based on the total squared error of that the model incurs on the task of predicting \emph{values} as determined by the last value estimate along the transitions. The next value function is then chosen by solving the optimistic planning problem with the constructed set of models. We derive a bound on the regret, which, in the special case of linear mixtures, the regret bound takes the form $\tilde{\mathcal{O}}(d\sqrt{H^{3}T})$, where $H$, $T$ and $d$ are the horizon, total number of steps and dimension of $\theta$, respectively. In particular, this regret bound is independent of the total number of states or actions, and is close to a lower bound

Author

Ayoub, Alex; Jia, Zeyu; Szepesvari, Csaba; Wang, Mengdi; Yang, Lin F.

Published

2020

Language

EN