Stochastic safety for Markov chains

Manuela L. Bujorianu; Rafael Wisniewski; Evangelos Boulougouris

doi:10.1109/LCSYS.2020.3002475

Stochastic safety for Markov chains

Manuela L. Bujorianu, Rafael Wisniewski, Evangelos Boulougouris

Naval Architecture, Ocean And Marine Engineering

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

18 Downloads (Pure)

Abstract

In this letter, we study the so-called p-safety of a Markov chain. We say that a state is p-safe in a state space S with respect to an unsafe set U if the process stays in the state space and hits the set U with the probability less than p. We show several ways of computing p-safety: by means the Dirichlet problem, the evolution equation, the barrier certificates, and the Martin kernel. The set of barrier certificates forms a cone. We show how to generate barrier certificates from the set of extreme points of a cone base.

Original language	English
Pages (from-to)	427-432
Number of pages	6
Journal	IEEE Control Systems Letters
Volume	5
Issue number	2
Early online date	15 Jun 2020
DOIs	https://doi.org/10.1109/LCSYS.2020.3002475
Publication status	Published - Apr 2021

Keywords

stochastic systems
computational methods
optimization
numerical algorithms
Lyapunov methods
Markov processes
algorithms

Access to Document

10.1109/LCSYS.2020.3002475

Bujorianu-etal-IEEE-CSL-2020-Stochastic-safety-for-Markov-chainsAccepted author manuscript, 466 KB

Cite this

@article{a81f179f090d40199e84fafa19fd9d5a,

title = "Stochastic safety for Markov chains",

abstract = "In this letter, we study the so-called p-safety of a Markov chain. We say that a state is p-safe in a state space S with respect to an unsafe set U if the process stays in the state space and hits the set U with the probability less than p. We show several ways of computing p-safety: by means the Dirichlet problem, the evolution equation, the barrier certificates, and the Martin kernel. The set of barrier certificates forms a cone. We show how to generate barrier certificates from the set of extreme points of a cone base.",

keywords = "stochastic systems, computational methods, optimization, numerical algorithms, Lyapunov methods, Markov processes, algorithms",

author = "Bujorianu, {Manuela L.} and Rafael Wisniewski and Evangelos Boulougouris",

note = "{\textcopyright} 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",

year = "2021",

month = apr,

doi = "10.1109/LCSYS.2020.3002475",

language = "English",

volume = "5",

pages = "427--432",

number = "2",

}

TY - JOUR

T1 - Stochastic safety for Markov chains

AU - Bujorianu, Manuela L.

AU - Wisniewski, Rafael

AU - Boulougouris, Evangelos

N1 - © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2021/4

Y1 - 2021/4

N2 - In this letter, we study the so-called p-safety of a Markov chain. We say that a state is p-safe in a state space S with respect to an unsafe set U if the process stays in the state space and hits the set U with the probability less than p. We show several ways of computing p-safety: by means the Dirichlet problem, the evolution equation, the barrier certificates, and the Martin kernel. The set of barrier certificates forms a cone. We show how to generate barrier certificates from the set of extreme points of a cone base.

AB - In this letter, we study the so-called p-safety of a Markov chain. We say that a state is p-safe in a state space S with respect to an unsafe set U if the process stays in the state space and hits the set U with the probability less than p. We show several ways of computing p-safety: by means the Dirichlet problem, the evolution equation, the barrier certificates, and the Martin kernel. The set of barrier certificates forms a cone. We show how to generate barrier certificates from the set of extreme points of a cone base.

KW - stochastic systems

KW - computational methods

KW - optimization

KW - numerical algorithms

KW - Lyapunov methods

KW - Markov processes

KW - algorithms

U2 - 10.1109/LCSYS.2020.3002475

DO - 10.1109/LCSYS.2020.3002475

M3 - Article

VL - 5

SP - 427

EP - 432

IS - 2

ER -