SMEDL: Combining Synchronous and Asynchronous Monitoring

TLDR: A hybrid approach is explored, where low-level properties are checked synchronously, while higher-level ones are checked asynchronously, in runtime verification of monitors based on an architecture specification.

Abstract: Two major approaches have emerged in runtime verification, based on synchronous and asynchronous monitoring. Each approach has its advantages and disadvantages and is applicable in different situations. In this paper, we explore a hybrid approach, where low-level properties are checked synchronously, while higher-level ones are checked asynchronously. We present a tool for constructing and deploying monitors based on an architecture specification. Monitor logic and patterns of communication between monitors are specified in a language SMEDL. The language and the tool are illustrated using a case study of a robotic simulator.

Figures

Fig. 1. SMEDL overview

Fig. 4. Execution time vs. number of threads

Fig. 3. Monitor Network of Explorer

Fig. 2. SMEDL Toolchain

Table 1. Relation between input size and overhead

Full text

University of Pennsylvania University of Pennsylvania
ScholarlyCommons ScholarlyCommons
Departmental Papers (CIS) Department of Computer & Information Science
9-2016
SMEDL: Combining Synchronous and Asynchronous Monitoring SMEDL: Combining Synchronous and Asynchronous Monitoring
Teng Zhang
University of Pennsylvania
, tengz@cis.upenn.edu
Peter Gebhard
University of Pennsylvania
, pgeb@cis.upenn.edu
Oleg Sokolsky
University of Pennsylvania
, sokolsky@cis.upenn.edu
Follow this and additional works at: https://repository.upenn.edu/cis_papers
Part of the Computer Engineering Commons, and the Computer Sciences Commons
Recommended Citation Recommended Citation
Teng Zhang, Peter Gebhard, and Oleg Sokolsky, "SMEDL: Combining Synchronous and Asynchronous
Monitoring",
The 16th Conference on Runtime VeriBcation (RV 2016)
, 482-490. September 2016.
The 16th Conference on Runtime VeriBcation (RV 2016), pp. 482--490. Madrid, Spain, September 2016.
This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cis_papers/825
For more information, please contact repository@pobox.upenn.edu.

SMEDL: Combining Synchronous and Asynchronous Monitoring SMEDL: Combining Synchronous and Asynchronous Monitoring
Abstract Abstract
Two major approaches have emerged in runtime veriBcation, based on synchronous and asynchronous
monitoring. Each approach has its advantages and disadvantages and is applicable in different
situations. In this paper, we explore a hybrid approach, where low-level properties are checked
synchronously, while higher-level ones are checked asynchronously. We present a tool for constructing
and deploying monitors based on an architecture speciBcation. Monitor logic and patterns of
communication between monitors are speciBed in a language SMEDL. The language and the tool are
illustrated using a case study of a robotic simulator.
Keywords Keywords
monitor generation, synchronous monitoring, asynchronous monitoring
Disciplines Disciplines
Computer Engineering | Computer Sciences
Comments Comments
The 16th Conference on Runtime VeriBcation (RV 2016), pp. 482--490. Madrid, Spain, September 2016.
This conference paper is available at ScholarlyCommons: https://repository.upenn.edu/cis_papers/825

SMEDL: Combining Synchronous and
Asynchronous Monitoring
Teng Zhang
1
, Peter Gebhard
1
, and Oleg Sokolsky
1
University of Pennsylvania, Philadelphia PA 19104, USA,
{tengz,pgeb,sokolsky}@cis.upenn.edu
Abstract. Two major approaches have emerged in runtime verification,
based on synchronous and asynchronous monitoring. Each approach has
its advantages and disadvantages and is applicable in different situations.
In this paper, we explore a hybrid approach, where low-level proper-
ties are checked synchronously, while higher-level ones are checked asyn-
chronously. We present a tool for constructing and deploying monitors
based on an architecture specification. Monitor logic and patterns of
communication between monitors are specified in a language SMEDL.
The language and the tool are illustrated using a case study of a robotic
simulator.
Keywords: Monitor generation, Synchronous monitoring, Asynchronous
monitoring
1 Introduction
Runtime verification(RV) [1] has emerged as a powerful technique for correct-
ness monitoring of critical systems. Numbers of approaches have been proposed
among which synchronous monitoring [2] and asynchronous monitoring [3] are
broadly used. Synchronous monitoring will block the execution of the system
being monitored until validity of an observation is confirmed, ensuring that po-
tentially hazardous behavior is not propagated to the system environment. This
makes this method suitable for safety- and security-related contexts. However,
synchronous monitoring incurs high execution overhead for the target system,
and less critical properties may not require such strict guarantees. On the other
hand, asynchronous monitoring may allow to check the properties with less over-
head for the target system, but as the system continues its execution while
checking is performed, it may not be suitable for some critical properties. More-
over, the error point is hard to locate: when the monitor reports the violation
of the property, the target system may have already left the position causing
the problem. Most RV tools target one of these two approaches. Furthermore,
synchronous monitoring may not be suitable for distributed systems. In many
practical cases, it is desirable to combine the two approaches to get the benefits
of both and reduce effects of drawbacks.
The contribution of this paper is a tool for construction and deployment
of hybrid monitoring. The tool uses the language SMEDL to specify monitoring

architecture and individual monitors. Properties to be checked are represented in
a state-machine style in monitors. Generated monitors can be integrated with the
target system or deployed separately, according to the architecture specification.
Execution within a monitor is synchronous while the communication among
monitors is asynchronous. This allows us to monitor properties on multiple time
scales and levels of criticality.
Related work. MaC(Monitoring and Checking) [4] is an architecture for
asynchronous runtime monitoring. A distributed version of MaC, DMaC [5],
is proposed mainly for monitoring the properties of network protocols. MOP
(Monitoring Oriented Programming) [6] is a generic framework for properties
specification and the checking of the properties at runtime. Based on the work
of MOP, RV-Monitor [7] can monitor hundreds of properties of Java API specifi-
cations at the same time. Using the concepts of AOP [8] and MOP, MOVEC [9]
is compiler supporting the parametric runtime verification for systems written
in C. [10] proposes an architecture allowing for switching between synchronous
and asynchronous monitoring but it is not clear how to use synchronous and
asynchronous monitoring simultaneously. [11] proposed PT-DTL, a temporal
logical to describe the temporal properties of distributed system, and a decen-
tralized monitoring algorithm for PT-DTL. However, the proposed tool, DIANA,
only supports the asynchronous monitoring for distributed program with a fixed
architecture. [12] presents a method for monitoring multi-threaded component-
based systems described in BIP but it is not suitable for distributed systems. [13]
proposes a primitive condition rule-based system RuleR which supports the hi-
erarchy architecture of monitors but asynchronous monitoring is not supported.
Thus, despite the variety of available tools, there is presently no support for
combining synchronous and asynchronous monitoring.
The paper is organized as follows. Section 2 gives a overview of SMEDL.
Section 3 introduces the implementation of the SMEDL tool. Section 4 uses the
case study of simple robot simulator to evaluate the performance of the tool.
Section 5 concludes the paper and presents the future work.
2 Overview of SMEDL
2.1 SMEDL concepts
A SMEDL monitoring system can be divided as four parts: target system, moni-
toring specification, SMEDL code generator and runtime checkers, as illustrated
in Fig. 1. Target system is the system to be monitored and checked. The SMEDL
specification contains a set of monitoring objects and an architecture that cap-
tures patterns of communication between them. A monitoring object can be an
abstraction of a system object or an abstract entity that represents interactions
between multiple system objects. Objects can include a set of parameters whose
values are fixed when the object is instantiated. Internal state can be maintained
in an object to reflect the history of its evolution. SMEDL events are instanta-
neous occurrences that ultimately originate from observations of the execution
of target system. Events can also be raised by monitors in response to other

events. Raised events can be delivered to other monitors for checking or serve as
alarms. A SMEDL specification is independent from the system implementation
so that the monitoring specification does not need to be changed as long as the
specification remains the same, even if the implementation has been changed.
Instead, the definition of events in terms of observations on the target system is
modified. Each monitoring object is converted to executable code by the SMEDL
code generator and can be instantiated multiple times with different parameters,
either statically during the target system startup or dynamically at run time,
in response to system events such as the creation of a new thread in the target
system.
Fig. 1. SMEDL overview
2.2 Brief description of the language
The SMEDL specification contains two parts: a definition for each monitor object
and a description of the monitor network that specifies monitor instances and
connections between them.
Monitoring objects. A SMEDL monitoring object is a collection of ex-
tended finite state machines (EFSMs) sharing a set of internal state variables and
events. More precisely, a monitoring object is a tuple hinterf ace, implementationi,
where interface contains the name, unchangeable identity parameters and event
declarations; implementation contains state variables and state machines of the
monitor. In SMEDL syntax, illustrated in the case study, state machines are rep-
resented as scenarios. Three kinds of events, imported, exported and internal,
can be specified in the event declaration. Imported events of a monitor can
be received and used to trigger the execution of the monitor; exported events
are raised in the monitor and are sent to other monitors; internal events are
processed within the monitor instance. State machines are used to define the
behavior of the monitor. Transitions of the state machines are labelled with
events that trigger the transition. In addition to an event, each transition may
also be labeled by a guard, which is a predicate over state variables and event
attributes, and a set of actions, each action is either an assignment to a local
variable, or a statement that raises an event. Semantics for single monitors deter-
mines macro-steps, that is, synchronous compositions of individual transitions

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "Smedl: combining synchronous and asynchronous monitoring" ?

In this paper, the authors explore a hybrid approach, where low-level properties are checked synchronously, while higher-level ones are checked asynchronously. The authors present a tool for constructing and deploying monitors based on an architecture specification. The language and the tool are illustrated using a case study of a robotic simulator. 

Q2. What are the future works in "Smedl: combining synchronous and asynchronous monitoring" ?

In their tool, extension to synchronous monitoring over a network would be a simple extension to consider in the future. Potentially, deployments could be mixed, however implementation of the monitor becomes substantially more complicated. The most surprising lesson from the case study, for us, was that the overhead of sending an event to a separate monitor can be larger than checking the event synchronously within the same monitor. 

Q3. What is the overhead of a synchronous monitoring system?

There are three sources of overhead for synchronous monitoring: instantiation of monitors, checking of observations, and communication. 

Q4. What is the overhead of asynchronous checking?

Increasing the size of the map tends to reduce overhead, since robots tend to move straight over longer distances on a larger map, without generating observations. 

Q5. What is a generic framework for properties specification and the checking of the properties at runtime?

MOP (Monitoring Oriented Programming) [6] is a generic framework for properties specification and the checking of the properties at runtime. 

Q6. What is the purpose of synchronous monitoring?

Synchronous monitoring will block the execution of the system being monitored until validity of an observation is confirmed, ensuring that potentially hazardous behavior is not propagated to the system environment. 

Q7. What is the way to monitor the properties of a distributed system?

the proposed tool, DIANA, only supports the asynchronous monitoring for distributed program with a fixed architecture. [12] presents a method for monitoring multi-threaded componentbased systems described in BIP but it is not suitable for distributed systems. [13] proposes a primitive condition rule-based system RuleR which supports the hierarchy architecture of monitors but asynchronous monitoring is not supported. 

Q8. What is the architecture description language used for asynchronous communication between monitors?

To support the asynchronous communication between monitors, the authors use the publish-subscribe mechanism of the RabbitMQ middleware [16]. 

Q9. what is the function that is used to check the direction of the robot?

If the helper function check retrieved returns true, the exported event retrieved will be raised carrying the number of moves that the robot has taken to retrieve this target. 

Q10. What is the architecture description language for integrating monitor instances and target program?

The monitor generator generates the code for a single monitor object, while the configurator is responsible for integrating monitor instances and target program, based on the SMEDL architecture specification. 

Q11. What is the definition of a monitoring object?

Each monitoring object is converted to executable code by the SMEDL code generator and can be instantiated multiple times with different parameters, either statically during the target system startup or dynamically at run time, in response to system events such as the creation of a new thread in the target system. 

Q12. What is the disadvantage of synchronous monitoring?

synchronous monitoring incurs high execution overhead for the target system, and less critical properties may not require such strict guarantees. 

Q13. What is the definition of a monitoring system?

A SMEDL monitoring system can be divided as four parts: target system, monitoring specification, SMEDL code generator and runtime checkers, as illustrated in Fig. 

Q14. What is the overhead of asynchronous monitoring?

Communication overhead is incurred only when asynchronousmonitors are present and events need to be sent to the asynchronous monitors via the middleware. 

Q15. What is the language used for defining connection patterns between monitor instances?

Apart from the source and target monitoring objects and events, connection patterns also specify matching rules between source and target monitor instances according to instance parameters or attributes of the event. [15] gives a detailed description of the language. 

Q16. What is the surprising lesson from the case study?

The most surprising lesson from the case study, for us, was that the overhead of sending an event to a separate monitor can be larger than checking the event synchronously within the same monitor.