Can I read SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasets on EtoBox?

SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasets by Wan, Shenghua; Chen, Ziyuan; Gan, Le; Feng, Shuai; Zhan, De-Chuan is a scholarly article available to read on EtoBox.

What is SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasets about?

Model-based offline reinforcement Learning (RL) is a promising approach that leverages existing data effectively in many real-world applications, especially those involving high-dimensional inputs like images and videos. To alleviate the distribution shift issue in offline RL, existing model-based methods heavily rely on the uncertainty of learned dynamics. However, the model uncertainty estimation becomes significantly biased when observations contain complex distractors with non-trivial dynamics. To address this challenge, we propose a new approach - \emph{Separated Model-based Offline Policy Optimization} (SeMOPO) - decomposing latent states into endogenous and exogenous parts via conservative sampling and estimating model uncertainty on the endogenous states only. We provide a theoretical guarantee of model uncertainty and performance bound of SeMOPO. To assess the efficacy, we construct the Low-Quality Vision Deep Data-Driven Datasets for RL (LQV-D4RL), where the data are collected by non-expert policy and the observations include moving distractors. Experimental results show that our method substantially outperforms all baseline methods, and further analytical experiments val

Author

Wan, Shenghua; Chen, Ziyuan; Gan, Le; Feng, Shuai; Zhan, De-Chuan

Published

2024

Language

EN