Can I read VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion on EtoBox?

VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion by Zhu, Shaoting; Mou, Linzhan; Li, Derun; Ye, Baijun; Huang, Runhan; Zhao, Hang is a scholarly article available to read on EtoBox.

What is VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion about?

Recent success in legged robot locomotion is attributed to the integration of reinforcement learning and physical simulators. However, these policies often encounter challenges when deployed in real-world environments due to sim-to-real gaps, as simulators typically fail to replicate visual realism and complex real-world geometry. Moreover, the lack of realistic visual rendering limits the ability of these policies to support high-level tasks requiring RGB-based perception like ego-centric navigation. This paper presents a Real-to-Sim-to-Real framework that generates photorealistic and physically interactive "digital twin" simulation environments for visual navigation and locomotion learning. Our approach leverages 3D Gaussian Splatting (3DGS) based scene reconstruction from multi-view images and integrates these environments into simulations that support ego-centric visual perception and mesh-based physical interactions. To demonstrate its effectiveness, we train a reinforcement learning policy within the simulator to perform a visual goal-tracking task. Extensive experiments show that our framework achieves RGB-only sim-to-real policy transfer. Additionally, our framework facilit

Author

Zhu, Shaoting; Mou, Linzhan; Li, Derun; Ye, Baijun; Huang, Runhan; Zhao, Hang

Published

2025

Language

EN