Can I read Supervised Off-Policy Ranking on EtoBox?

Supervised Off-Policy Ranking by Jin, Yue; Zhang, Yue; Qin, Tao; Zhang, Xudong; Yuan, Jian; Li, Houqiang; Liu, Tie-Yan is a scholarly article available to read on EtoBox.

What is Supervised Off-Policy Ranking about?

Off-policy evaluation (OPE) is to evaluate a target policy with data generated by other policies. Most previous OPE methods focus on precisely estimating the true performance of a policy. We observe that in many applications, (1) the end goal of OPE is to compare two or multiple candidate policies and choose a good one, which is a much simpler task than precisely evaluating their true performance; and (2) there are usually multiple policies that have been deployed to serve users in real-world systems and thus the true performance of these policies can be known. Inspired by the two observations, in this work, we study a new problem, supervised off-policy ranking (SOPR), which aims to rank a set of target policies based on supervised learning by leveraging off-policy data and policies with known performance. We propose a method to solve SOPR, which learns a policy scoring model by minimizing a ranking loss of the training policies rather than estimating the precise policy performance. The scoring model in our method, a hierarchical Transformer based model, maps a set of state-action pairs to a score, where the state of each pair comes from the off-policy data and the action is taken

Author

Jin, Yue; Zhang, Yue; Qin, Tao; Zhang, Xudong; Yuan, Jian; Li, Houqiang; Liu, Tie-Yan

Published

2021

Language

EN