Can I read Variational Latent Branching Model for Off-Policy Evaluation on EtoBox?

Variational Latent Branching Model for Off-Policy Evaluation by Gao, Qitong; Gao, Ge; Chi, Min; Pajic, Miroslav is a scholarly article available to read on EtoBox.

What is Variational Latent Branching Model for Off-Policy Evaluation about?

Model-based methods have recently shown great potential for off-policy evaluation (OPE); offline trajectories induced by behavioral policies are fitted to transitions of Markov decision processes (MDPs), which are used to rollout simulated trajectories and estimate the performance of policies. Model-based OPE methods face two key challenges. First, as offline trajectories are usually fixed, they tend to cover limited state and action space. Second, the performance of model-based methods can be sensitive to the initialization of their parameters. In this work, we propose the variational latent branching model (VLBM) to learn the transition function of MDPs by formulating the environmental dynamics as a compact latent space, from which the next states and rewards are then sampled. Specifically, VLBM leverages and extends the variational inference framework with the recurrent state alignment (RSA), which is designed to capture as much information underlying the limited training data, by smoothing out the information flow between the variational (encoding) and generative (decoding) part of VLBM. Moreover, we also introduce the branching architecture to improve the model's robustness ag

Author

Gao, Qitong; Gao, Ge; Chi, Min; Pajic, Miroslav

Published

2023

Language

EN