Can I read Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence on EtoBox?

Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence by Zhao, Feiran; You, Keyou is a scholarly article available to read on EtoBox.

What is Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence about?

In safety-critical applications, reinforcement learning (RL) needs to consider safety constraints. However, theoretical understandings of constrained RL for continuous control are largely absent. As a case study, this paper presents a cost-constrained LQR formulation, where a number of LQR costs with user-defined penalty matrices are subject to constraints. To solve it, we propose a policy gradient primal-dual method to find an optimal state feedback gain. Despite the non-convexity of the cost-constrained LQR problem, we provide a constructive proof for strong duality and a geometric interpretation of an optimal multiplier set. By proving that the concave dual function is Lipschitz smooth, we further provide convergence guarantees for the PG primal-dual method. Finally, we perform simulations to validate our theoretical findings.

Author

Zhao, Feiran; You, Keyou

Published

2024

Language

EN