MoGym: Using Formal Models for Training and Verifying Decision-making Agents
Open Access
- 6 August 2022
- book chapter
- Published by Springer Nature
- p. 430-443
Abstract
MoGym, is an integrated toolbox enabling the training and verification of machine-learned decision-making agents based on formal models, for the purpose of sound use in the real world. Given a formal representation of a decision-making problem in the JANI format and a reach-avoid objective, MoGym(a) enables training a decision-making agent with respect to that objective directly on the model using reinforcement learning (RL) techniques, and (b) it supports rigorous assessment of the quality of the induced decision-making agent by means of deep statistical model checking (DSMC). MoGymimplements the standard interface for training environments established by OpenAI Gym, thereby connecting to the vast body of existing work in the RL community. In return, it makes accessible the large set of existing JANI model checking benchmarks to machine learning research. It thereby contributes an efficient feedback mechanism for improving in particular reinforcement learning algorithms. The connective part is implemented on top of Momba. For the DSMC quality assurance of the learned decision-making agents, a variant of the statistical model checkermodesof the ModestToolsetis leveraged, which has been extended by two new resolution strategies for non-determinism when encountered during statistical evaluation.Keywords
This publication has 47 references indexed in Scilit:
- On-the-Fly Confluence Detection for Statistical Model CheckingLecture Notes in Computer Science, 2013
- A compositional modelling and analysis framework for stochastic hybrid systemsFormal Methods in System Design, 2012
- Stochastic Model Checking2007
- On Statistical Model Checking of Stochastic Systems2005
- Approximate Probabilistic Model Checking2004
- Numerical vs. Statistical Probabilistic Model Checking: An Empirical StudyLecture Notes in Computer Science, 2004
- Probabilistic Model Checking of Deadline Properties in the IEEE 1394 FireWire Root Contention ProtocolFormal Aspects of Computing, 2003
- Probabilistic Verification of Discrete Event Systems Using Acceptance Sampling2002
- Root Contention in IEEE 13941999
- Learning to act using real-time dynamic programmingArtificial Intelligence, 1995