Can I read Stronger Regret Bounds for Safe Online Reinforcement Learning in the Linear Quadratic Regulator on EtoBox?

Stronger Regret Bounds for Safe Online Reinforcement Learning in the Linear Quadratic Regulator by Schiffer, Benjamin; Janson, Lucas is a scholarly article available to read on EtoBox.

What is Stronger Regret Bounds for Safe Online Reinforcement Learning in the Linear Quadratic Regulator about?

Many practical applications of online reinforcement learning require the satisfaction of safety constraints while learning about the unknown environment. In this work, we study Linear Quadratic Regulator (LQR) learning with unknown dynamics, but with the additional constraint that the position must stay within a safe region for the entire trajectory with high probability. Unlike in previous works, we allow for both bounded and unbounded noise distributions and study stronger baselines of nonlinear controllers that are better suited for constrained problems than linear controllers. Due to these complications, we focus on 1-dimensional state- and action- spaces, however we also discuss how we expect the high-level takeaways can generalize to higher dimensions. Our primary contribution is the first $\tilde{O}_T(\sqrt{T})$-regret bound for constrained LQR learning, which we show relative to a specific baseline of non-linear controllers. We then prove that, for any non-linear baseline satisfying natural assumptions, $\tilde{O}_T(\sqrt{T})$-regret is possible when the noise distribution has sufficiently large support and $\tilde{O}_T(T^{2/3})$-regret is possible for any subgaussian noise

Author

Schiffer, Benjamin; Janson, Lucas

Published

2024

Language

EN