Can I read Active Offline Policy Selection on EtoBox?

Active Offline Policy Selection by Konyushkova, Ksenia; Chen, Yutian; Paine, Tom Le; Gulcehre, Caglar; Paduraru, Cosmin; Mankowitz, Daniel J; Denil, Misha; de Freitas, Nando is a scholarly article available to read on EtoBox.

What is Active Offline Policy Selection about?

This paper addresses the problem of policy selection in domains with abundant logged data, but with a restricted interaction budget. Solving this problem would enable safe evaluation and deployment of offline reinforcement learning policies in industry, robotics, and recommendation domains among others. Several off-policy evaluation (OPE) techniques have been proposed to assess the value of policies using only logged data. However, there is still a big gap between the evaluation by OPE and the full online evaluation. Yet, large amounts of online interactions are often not possible in practice. To overcome this problem, we introduce active offline policy selection - a novel sequential decision approach that combines logged data with online interaction to identify the best policy. We use OPE estimates to warm start the online evaluation. Then, in order to utilize the limited environment interactions wisely we decide which policy to evaluate next based on a Bayesian optimization method with a kernel that represents policy similarity. We use multiple benchmarks, including real-world robotics, with a large number of candidate policies to show that the proposed approach improves upon sta

Author

Konyushkova, Ksenia; Chen, Yutian; Paine, Tom Le; Gulcehre, Caglar; Paduraru, Cosmin; Mankowitz, Daniel J; Denil, Misha; de Freitas, Nando

Published

2021

Language

EN