Stable and safe reinforcement learning via a barrier-Lyapunov actor-critic approach

Zhao, Liqun, Gatsis, Konstantinos and Papachristodoulou, Antonis (2024) Stable and safe reinforcement learning via a barrier-Lyapunov actor-critic approach. In Proceedings of the IEEE Conference on Decision and Control. pp. 1320-1325 . (doi:10.1109/CDC49753.2023.10383742).

Record type: Conference or Workshop Item (Paper)

Abstract

Reinforcement Learning (RL) has demonstrated impressive performance in various areas such as video games and robotics. However, ensuring safety and stability, which are two critical properties from a control perspective, remains a significant challenge when using RL to control real-world systems. In this paper, we first provide definitions of safety and stability for the RL system, and then combine the Control Barrier Function (CBF) and Control Lyapunov Function (CLF) methods with the actor-critic method in RL to propose a Barrier-Lyapunov Actor-Critic (BLAC) framework which helps maintain the aforementioned safety and stability for the system. In this framework, CBF constraints for safety and CLF constraint for stability are constructed based on the data sampled from the replay buffer, and the augmented Lagrangian method is used to update the parameters of the RL-based controller. Furthermore, an additional backup controller is introduced in case the RL-based controller cannot provide valid control signals w

This record has no associated files available for download.