Learning provably stabilizing neural controllers for discrete-time stochastic systems

We consider the problem of learning control policies in discrete-time stochastic systems which guarantee that the system stabilizes within some specified stabilization region with probability 1. Our approach is based on the novel notion of stabilizing ranking supermartingales (sRSMs) that we introdu...

Full description

Saved in:
Bibliographic Details
Main Authors: ANSARIPOUR, Matin, CHATTERJEE, Krishnendu, HENZINGER, A. Thomas, LECHNER, Mathias, ZIKELIC, Dorde
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/9067
https://ink.library.smu.edu.sg/context/sis_research/article/10070/viewcontent/978_3_031_45329_8.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-10070
record_format dspace
spelling sg-smu-ink.sis_research-100702024-08-01T15:26:15Z Learning provably stabilizing neural controllers for discrete-time stochastic systems ANSARIPOUR, Matin CHATTERJEE, Krishnendu HENZINGER, A. Thomas LECHNER, Mathias ZIKELIC, Dorde We consider the problem of learning control policies in discrete-time stochastic systems which guarantee that the system stabilizes within some specified stabilization region with probability 1. Our approach is based on the novel notion of stabilizing ranking supermartingales (sRSMs) that we introduce in this work. Our sRSMs overcome the limitation of methods proposed in previous works whose applicability is restricted to systems in which the stabilizing region cannot be left once entered under any control policy. We present a learning procedure that learns a control policy together with an sRSM that formally certifies probability 1 stability, both learned as neural networks. We show that this procedure can also be adapted to formally verifying that, under a given Lipschitz continuous control policy, the stochastic system stabilizes within some stabilizing region with probability 1. Our experimental evaluation shows that our learning procedure can successfully learn provably stabilizing policies in practice. 2023-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/9067 info:doi/10.1007/978-3-031-45329-8_17 https://ink.library.smu.edu.sg/context/sis_research/article/10070/viewcontent/978_3_031_45329_8.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Learning-based control Stochastic systems Martingales Formal verification Stabilization Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Learning-based control
Stochastic systems
Martingales
Formal verification
Stabilization
Databases and Information Systems
spellingShingle Learning-based control
Stochastic systems
Martingales
Formal verification
Stabilization
Databases and Information Systems
ANSARIPOUR, Matin
CHATTERJEE, Krishnendu
HENZINGER, A. Thomas
LECHNER, Mathias
ZIKELIC, Dorde
Learning provably stabilizing neural controllers for discrete-time stochastic systems
description We consider the problem of learning control policies in discrete-time stochastic systems which guarantee that the system stabilizes within some specified stabilization region with probability 1. Our approach is based on the novel notion of stabilizing ranking supermartingales (sRSMs) that we introduce in this work. Our sRSMs overcome the limitation of methods proposed in previous works whose applicability is restricted to systems in which the stabilizing region cannot be left once entered under any control policy. We present a learning procedure that learns a control policy together with an sRSM that formally certifies probability 1 stability, both learned as neural networks. We show that this procedure can also be adapted to formally verifying that, under a given Lipschitz continuous control policy, the stochastic system stabilizes within some stabilizing region with probability 1. Our experimental evaluation shows that our learning procedure can successfully learn provably stabilizing policies in practice.
format text
author ANSARIPOUR, Matin
CHATTERJEE, Krishnendu
HENZINGER, A. Thomas
LECHNER, Mathias
ZIKELIC, Dorde
author_facet ANSARIPOUR, Matin
CHATTERJEE, Krishnendu
HENZINGER, A. Thomas
LECHNER, Mathias
ZIKELIC, Dorde
author_sort ANSARIPOUR, Matin
title Learning provably stabilizing neural controllers for discrete-time stochastic systems
title_short Learning provably stabilizing neural controllers for discrete-time stochastic systems
title_full Learning provably stabilizing neural controllers for discrete-time stochastic systems
title_fullStr Learning provably stabilizing neural controllers for discrete-time stochastic systems
title_full_unstemmed Learning provably stabilizing neural controllers for discrete-time stochastic systems
title_sort learning provably stabilizing neural controllers for discrete-time stochastic systems
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/9067
https://ink.library.smu.edu.sg/context/sis_research/article/10070/viewcontent/978_3_031_45329_8.pdf
_version_ 1814047722395664384