Showing papers by "Joost-Pieter Katoen published in 2015"

PROPhESY : A PRObabilistic ParamEter SYnthesis Tool

Abstract: We present PROPhESY, a tool for analyzing parametric Markov chains (MCs). It can compute a rational function (i.e., a fraction of two polynomials in the model parameters) for reachability and expected reward objectives. Our tool outperforms state-of-the-art tools and supports the novel feature of conditional probabilities. PROPhESY supports incremental automatic parameter synthesis (using SMT techniques) to determine “safe” and “unsafe” regions of the parameter space. All values in these regions give rise to instantiated MCs satisfying or violating the (conditional) probability or expected reward objective. PROPhESY features a web front-end supporting visualization and user-guided parameter synthesis. Experimental results show that PROPhESY scales to MCs with millions of states and several parameters. Open image in new window

A Greedy Approach for the Efficient Repair of Stochastic Models

Abstract: For discrete-time probabilistic models there are efficient methods to check whether they satisfy certain properties. If a property is refuted, available techniques can be used to explain the failure in form of a counterexample. However, there are no scalable approaches to repair a model, i.e., to modify it with respect to certain side conditions such that the property is satisfied. In this paper we propose such a method, which avoids expensive computations and is therefore applicable to large models. A prototype implementation is used to demonstrate the applicability and scalability of our technique.

On the Hardness of Almost–Sure Termination

Abstract: This paper considers the computational hardness of computing expected outcomes and deciding (universal) (positive) almost–sure termination of probabilistic programs. It is shown that computing lower and upper bounds of expected outcomes is \(\varSigma _1^0\)– and \(\varSigma _2^0\)–complete, respectively. Deciding (universal) almost–sure termination as well as deciding whether the expected outcome of a program equals a given rational value is shown to be \(\varPi ^0_2\)–complete. Finally, it is shown that deciding (universal) positive almost–sure termination is \(\varSigma _2^0\)–complete (\(\varPi _3^0\)–complete).

Safety-Constrained Reinforcement Learning for MDPs

Abstract: We consider controller synthesis for stochastic and partially unknown environments in which safety is essential. Specifically, we abstract the problem as a Markov decision process in which the expected performance is measured using a cost function that is unknown prior to run-time exploration of the state space. Standard learning approaches synthesize cost-optimal strategies without guaranteeing safety properties. To remedy this, we first compute safe, permissive strategies. Then, exploration is constrained to these strategies and thereby meets the imposed safety requirements. Exploiting an iterative learning procedure, the resulting policy is safety-constrained and optimal. We show correctness and completeness of the method and discuss the use of several heuristics to increase its scalability. Finally, we demonstrate the applicability by means of a prototype implementation.

On the Hardness of Almost-Sure Termination

Abstract: This paper considers the computational hardness of computing expected outcomes and deciding (universal) (positive) almost-sure termination of probabilistic programs. It is shown that computing lower and upper bounds of expected outcomes is $\Sigma_1^0$- and $\Sigma_2^0$-complete, respectively. Deciding (universal) almost-sure termination as well as deciding whether the expected outcome of a program equals a given rational value is shown to be $\Pi^0_2$-complete. Finally, it is shown that deciding (universal) positive almost-sure termination is $\Sigma_2^0$-complete ($\Pi_3^0$-complete).

Understanding Probabilistic Programs

Abstract: We present two views of probabilistic programs and their relationship. An operational interpretation as well as a weakest pre-condition semantics are provided for an elementary probabilistic guarded command language. Our study treats important features such as sampling, conditioning, loop divergence, and non-determinism.

Fault Trees on a Diet

Abstract: Fault trees are a popular industrial technique for reliability modelling and analysis. Their extension with common reliability patterns, such as spare management, functional dependencies, and sequencing -- known as dynamic fault trees DFTs -- has an adverse effect on scalability, prohibiting the analysis of complex, industrial cases by, e.g., probabilistic model checkers. This paper presents a novel, fully automated reduction technique for DFTs. The key idea is to interpret DFTs as directed graphs and exploit graph rewriting to simplify them. We present a collection of rewrite rules, address their correctness, and give a simple heuristic to determine the order of rewriting. Experiments on a large set of benchmarks show substantial DFT simplifications, yielding state space reductions and timing gains of up to two orders of magnitude.

Conditioning in Probabilistic Programming

Abstract: We investigate the semantic intricacies of conditioning, a main feature in probabilistic programming. We provide a weakest (liberal) pre-condition (w(l)p) semantics for the elementary probabilistic programming language pGCL extended with conditioning. We prove that quantitative weakest (liberal) pre-conditions coincide with conditional (liberal) expected rewards in Markov chains and show that semantically conditioning is a truly conservative extension. We present two program transformations which entirely eliminate conditioning from any program and prove their correctness using the w(l)p-semantics. Finally, we show how the w(l)p-semantics can be used to determine conditional probabilities in a parametric anonymity protocol and show that an inductive w(l)p-semantics for conditioning in non-deterministic probabilistic programs cannot exist.

Model Checking of Open Interval Markov Chains

Abstract: We consider the model checking problem for interval Markov chains with open intervals. Interval Markov chains are generalizations of discrete time Markov chains where the transition probabilities are intervals, instead of constant values. We focus on the case where the intervals are open. At first sight, open intervals present technical challenges, as optimal (min, max) value for reachability may not exist. We show that, as far as model checking (and reachability) is concerned, open intervals does not cause any problem, and with minor modification existing algorithms can be used for model checking interval Markov chains against PCTL formulas.

High-level Counterexamples for Probabilistic Automata

Abstract: Providing compact and understandable counterexamples for violated system properties is an essential task in model checking. Existing works on counterexamples for probabilistic systems so far computed either a large set of system runs or a subset of the system's states, both of which are of limited use in manual debugging. Many probabilistic systems are described in a guarded command language like the one used by the popular model checker PRISM. In this paper we describe how a smallest possible subset of the commands can be identified which together make the system erroneous. We additionally show how the selected commands can be further simplified to obtain a well-understandable counterexample.

Multi-objective Parameter Synthesis in Probabilistic Hybrid Systems

Abstract: Technical systems interacting with the real world can be elegantly modelled using probabilistic hybrid automata (PHA). Parametric probabilistic hybrid automata are dynamical systems featuring hybrid discrete-continuous dynamics and parametric probabilistic branching, thereby generalizing PHA by capturing a family of PHA within a single model. Such system models have a broad range of applications, from control systems over network protocols to biological components. We present a novel method to synthesize parameter instances (if such exist) of PHA satisfying a multi-objective bounded horizon specification over expected rewards. Our approach combines three techniques: statistical model checking of model instantiations, a symbolic version of importance sampling to handle the parametric dependence, and SAT-modulo-theory solving for finding feasible parameter instances in a multi-objective setting. The method provides statistical guarantees on the synthesized parameter instances. To illustrate the practical feasibility of the approach, we present experiments showing the potential benefit of the scheme compared to a naive parameter exploration approach.

A Statistical Approach for Timed Reachability in AADL Models

Abstract: We introduce a simulator (slimsim) for a subset of AADL extended with formalized behavioral semantics for nominal and error models The simulator allows to perform probabilistic analysis using the Monte Carlo method, on linear-hybrid, stochastic models, which describe a combination of nominal and error behaviors of hard- and software components The tool supports the use of different strategies, which control the behavior of the simulator when dealing with various forms of non-determinism The simulator is tested using benchmarks of the COMPASS toolset, as well as a case study by Airbus Defense and Space

Juggrnaut: using graph grammars for abstracting unbounded heap structures

Abstract: This paper presents a novel abstraction framework for heap data structures. It employs graph grammars, more precisely context-free hyperedge replacement grammars. We will show that this is a very natural formalism for modelling dynamic data structures in an intuitive way. Our approach aims at extending finite-state verification techniques to handle pointer-manipulating programs operating on complex dynamic data structures that are potentially unbounded in their size. The theoretical foundations of our approach and its correctness are the main focus of this paper. In addition, we present a prototypical tool entitled Juggrnaut that realizes our approach and show encouraging experimental verification results for three case studies: a doubly-linked list reversal, the flattening of binary trees, and the Deutsch---Schorr---Waite tree traversal algorithm.

Counterexamples for Expected Rewards

Abstract: The computation of counterexamples for probabilistic systems has gained a lot of attention during the last few years. All of the proposed methods focus on the situation when the probabilities of certain events are too high. In this paper we investigate how counterexamples for properties concerning expected costs (or, equivalently, expected rewards) of events can be computed. We propose methods to extract a minimal subsystem which already leads to costs beyond the allowed bound. Besides these exact methods, we present heuristic approaches based on path search and on best-first search, which are applicable to very large systems when deriving a minimum subsystem becomes infeasible due to the system size. Experiments show that we can compute counterexamples for systems with millions of states.

Modelling and statistical model checking of a microgrid

Abstract: This paper reports on the modelling and analysis of a microgrid with wind, microturbines, and the main grid as generation resources. The microgrid is modelled as a parallel composition of various stochastic hybrid automata. Extensive simulation runs of the behaviour of the main individual microgrid components give insight into the complex dynamics of the system and provide useful information to determine adequate parameter settings. The analysis of the microgrid focuses on determining the probability of linear temporal logic properties expressed in the logic LTL, using the statistical model checker Uppaal-SMC.

Reduction Techniques for Nondeterministic and Probabilistic Systems

Abstract: Model checking is an automated verification method guaranteeing that a mathematical model of a system satisfies a formally described property. It can be used to assess both qualitative and quantitative properties of complex software and hardware systems. Model checking suffers from the well-known state space explosion problem where the number of states grows exponentially in the number of program variables, channels and parallel components. Reduction techniques can be used to shrink the state space of system models by hiding redundant information and removing irrelevant details. The reduced state space can then be used for analysis provided it preserves a rich class of properties of interest. This thesis presents reduction techniques for a wide range of nondeterministic and probabilistic models. Our reduction techniques are based on the notions of equivalence relations and layering. Equivalence relations reduce the state space of system models, by aggregating equivalent states into a single state. The reduced state space obtained under an equivalence relation, is called a quotient system. An example equivalence relation that is widely used to reduce the state space of nondeterministic and probabilistic models is bisimulation. On the other hand, layering involves carrying out structural transformations for the systems that are modeled as a network of system models, e.g., distributed systems. As a result of these structural transformations, the new state space obtained is smaller than the original non-layered one. The first part of this thesis focuses on developing new equivalence relations for nondeterministic and Markovian models. For each of these relations, we define a quotient system, investigate its relationship with bisimulation and prove that it preserves interesting linear-time properties. In the second part of this thesis we focus on layering based state space reduction for more expressive specification formalisms that support a stepwise refinement methodology. We develop a framework of layering for modal transition systems and probabilistic versions thereof. This involves a layered composition

Probabilistic Programming: A True Verification Challenge

Abstract: Probabilistic programs [6] are sequential programs, written in languages like C, Java, Scala, or ML, with two added constructs: (1) the ability to draw values at random from probability distributions, and (2) the ability to condition values of variables in a program through observations. For a comprehensive treatment, see [3]. They have a wide range of applications. Probabilistic programming is at the heart of machine learning for describing distribution functions; Bayesian inference is pivotal in their analysis. Probabilistic programs are central in security for describing cryptographic constructions (such as randomised encryption) and security experiments. In addition, probabilistic programs are an active research topic in quantitative information flow. Quantum programs are inherently probabilistic due to the random outcomes of quantum measurements. Finally, probabilistic programs can be used for approximate computing, e.g., by specifying reliability requirements for programs that allocate data in unreliable memory and use unreliable operations in hardware (so as to save energy dissipation) [1]. Other applications include [4] scientific modeling, information retrieval, bio–informatics, epidemiology, vision, seismic analysis, semantic web, business intelligence, human cognition, and more. Microsoft has started an initiative to improve the usability of probabilistic programming which has resulted in languages such as R2 [13] and Tabular [5] emerged.

Abstraction and refinement of probabilistic automata using modal stochastic games

Abstract: ion and Refinement of Probabilistic Automata using Modal Stochastic Games

Challenges and Trends in Probabilistic Programming (Dagstuhl Seminar 15181)

Abstract: This report documents the program and the outcomes of Dagstuhl Seminar 15181 "Challenges and Trends in Probabilistic Programming". Probabilistic programming is at the heart of machine learning for describing distribution functions; Bayesian inference is pivotal in their analysis. Probabilistic programs are used in security for describing both cryptographic constructions (such as randomised encryption) and security experiments. In addition, probabilistic models are an active research topic in quantitative information now. Quantum programs are inherently probabilistic due to the random outcomes of quantum measurements. Finally, there is a rapidly growing interest in program analysis of probabilistic programs, whether it be using model checking, theorem proving, static analysis, or similar. Dagstuhl Seminar 15181 brought researchers from these various research communities together so as to exploit synergies and realize cross-fertilisation.

On the Hardness of PCTL Satisfiability

Abstract: This paper shows that the satisfiability problem for probabilistic CTL (PCTL, for short) is undecidable. By a reduction from $1\frac{1}{2}$-player games with PCTL winning objectives, we establish that the PCTL satisfiability problem is ${\Sigma}_1^1$-hard. We present an exponential-time algorithm for the satisfiability of a bounded, negation-closed fragment of PCTL, and show that the satisfiability problem for this fragment is EXPTIME-hard.

High-level counterexamples

Abstract: Providing compact and understandable counterexamples for violated system properties is an essential task in model checking. Existing works on counterexamples for probabilistic systems so far computed either a large set of system runs or a subset of the system's states, both of which are of limited use in manual debugging. Many probabilistic systems are described in a guarded command language like the one used by the popular model checker PRISM. In this paper we describe how a smallest possible subset of the commands can be identified which together make the system erroneous. We additionally show how the selected commands can be further simplified to obtain a well-understandable counterexample.

Probabilistic Programs - A Natural Model for Approximate Computations

Abstract: Approximate computing is a computational paradigm in which the requirement of having fully deterministic and accurate arithmetical operations is dropped [4]. For diverse practical problem (e.g. video decoding or image compression) by relaxing this requirement one can obtain considerably more efficient algorithms—be it in terms of execution time, memory usage, energy consumption, etc. When moving from accurate to approximate computations, a natural question arises: How accurate is the result of the approximate computation compared to the result of the accurate one? Any attempt to seriously answer this question raises the need for an appropriate formal model of approximate computations. Probabilistic programs extend conventional programs with probabilistic choices or random assignments and indeed provide a well–studied and suitable model for this purpose. Their syntax and semantics [5, 6, 3, 2] are flexible and expressive enough to model many aspects of approximate computing quite naturally. A recent approach for dealing with quantitative reliability analysis of approximate computations brought up the concept of unreliable operators [1]. In this approach instead of performing e.g. the accurate assignment x = x + y, one performs the approximate assignment x = x +. y, the latter being specified through a reliability value 0 ≤ e ≤ 1. Then with probability 1−e the approximate assignment yields the expected effect, while with probability e it produces an arbitrary—and thus erroneous—effect. This model, however, is unable to quantify how much the erroneous effect departs from the accurate one. Our goal is to take this reliability analysis a step further by employing a more expressive language of probabilistic programs: the so-called probabilistic guarded command language (pGCL) [6]—an extension of Dijkstra’s guarded command language with probabilistic choices. By employing this language besides representing the probability with wich erroneous computation occurs, we can provide more stochastic information such as the most likely effect or the expected (i.e. average) effect of the erroneous computations. For instance we can model an approximate assignment as a pGCL program