City, University of London Institutional Repository
Citation: Kloukinas, C. and Yovine, S. (2003). Synthesis of Safe, QoS Extendible,
Application Specific Schedulers for Heterogeneous Real-Time Systems. Paper presented at
the 15th Euromicro Conference on Real-Time Systems, 2003, Porto, Portugal.
This is the unspecified version of the paper.
This version of the publication may differ from the final published
version.
Permanent repository link: https://openaccess.city.ac.uk/id/eprint/2897/
Link to published version: http://dx.doi.org/10.1109/EMRTS.2003.1212754
Copyright: City Research Online aims to make research outputs of City,
University of London available to a wider audience. Copyright and Moral
Rights remain with the author(s) and/or copyright holders. URLs from
City Research Online may be freely distributed and linked to.
Reuse: Copies of full items can be used for personal research or study,
educational, or not-for-profit purposes without prior permission or
charge. Provided that the authors, title and full bibliographic details are
credited, a hyperlink and/or URL is given for the original metadata page
and the content is not changed in any way.
City Research Online: http://openaccess.city.ac.uk/ publications@city.ac.uk
City Research Online
Synthesis of Safe, QoS Extendible, Application Specific Schedulers for
Heterogeneous Real-Time Systems
∗
Christos KLOUKINAS Sergio YOVINE
VERIMAG, Centre
´
Equation, 2 avenue de Vignate, 38610 GI
`
ERES, France
E-mail: Christos.Kloukinas@imag.fr & Sergio.Yovine@imag.fr
Abstract
We present a new scheduler architecture, which permits
adding QoS policies to the scheduling decisions. We also
present a new scheduling synthesis method which allows
a designer to obtain a safe scheduler for a particular ap-
plication. Our scheduler architecture and scheduler syn-
thesis method can be used for heterogeneous applications
where the tasks communicate through various synchronisa-
tion primitives. We present a prototype implementation of
this scheduler architecture and related mechanisms on top
of an open-source OS for embedded systems.
1. Introduction
Safety & mission-critical systems need to be of ex-
tremely high quality, due to the great dangers and the high
cost of their potential failure. For this reason, when they
are multi-threaded they must be guaranteed to be free of
deadlocks and all threads must be guaranteed to meet their
deadlines under all circumstances.
The current practice for avoiding deadlocks is to use the
immediate priority ceiling protocol (IPCP) [8] for the shar-
ing of non-preemptable resources. This approach has a cer-
tain number of disadvantages though. All the priority in-
heritance family of protocols are pessimistic in nature and,
therefore, can refuse access to a shared resource even when
there is no real danger of a deadlock at the current situa-
tion. Additionally, in order to apply the IPCP, the designer
of the system must choose a set of priorities for all the tasks
in the system. Furthermore, for each shared resource the
designer must identify the threads which use it, in order to
assign a priority to that resource (i.e., the ceiling priority of
the resource). More importantly, however, the IPCP cannot
on its own support tasks which synchronise using monitors
and communicate using condition variables, as for exam-
ple is done in JAVA [3]. In such cases one should split tasks,
which means that the complexity of what the designers have
∗
Partially funded by the French RNTL project Expresso.
to do in order to use IPCP is quite high. Worse yet, not all
cases of communication through condition variables can be
translated according to these rules, since there can be tasks
which wait on a condition variable while still holding some
resource locked. These tasks cannot be split since the un-
derlying assumptions of the IPCP clearly demand that tasks
finish executing without holding any resources. Therefore,
it is not straightforward how one can use the IPCP with a
language such as R-T JAVA [9].
Besides, the methods currently used do not allow de-
signers to easily extend them for incorporating QoS to the
scheduler decisions. Being able to extend a scheduler with
QoS characteristics could allow us to experiment with ways
to minimise energy consumption, or further increase the
speed of the system, by minimising, for example, the num-
ber of context switches. An example of this can be found in
[2] where the authors present a dynamic scheduling method
which also treats the QoS aspect of the system. However, in
this work the authors consider a fixed task model where all
tasks are periodic and do not consider deadlock situations
or the communication aspect of the system.
In the following, we present a method for synthesising
QoS extendible and safe schedulers following the controller
synthesis paradigm and continuing previous work at Ver-
imag [1]. We start in section 2 by presenting the overall
architecture of the scheduler. Then, in section 3 we present
the model of the systems we consider and in section 4 the
particular method we use for the synthesis of the scheduler.
In section 5 we present a prototype implementation of our
scheduler on top of a real operating system and we conclude
in section 6.
2. Scheduler Architecture
We consider a set of threads synchronising through mon-
itors and communicating through condition variables. All
threads and shared objects are created at the initialisation
phase. We only consider applications executing on a single
processor for the moment. The architecture of the sched-
ulers we synthesise consists of two three-layered stacks, as
shown in Figure 1.
The left stack is responsible for selecting an application
thread for execution. The right stack is responsible for se-
lecting an application thread for the reception of a notifica-
tion. This stack allows the scheduler to control the commu-
nication aspect of the system. Being able to control which
thread will be notified for a particular event is something
that other scheduling policies like the PCP do not offer,
since they concentrate only on the selection of threads for
execution.
After one of the scheduler stacks is finished, it passes
control to an underlying R-T OS which provides low-level
kernel mechanisms. Such mechanisms include the ability to
create, suspend and resume an application thread, as well as
the ability to create, set and disable alarms for future events
(e.g., arrival of next period or the timeout of a
waitTimed
).
2.1. Controlling the Currently Executing Thread
The left stack takes control of the system when the
application calls one of
monitorEnter
,
monitorExit
,
waitForPeriod
,
wait
and
waitTimed
, or when an alarm
expires. In these cases, it must choose one of the avail-
able threads as the thread which should next be run on the
processor. It does this in three steps, each one performed at
a different layer.
In the first layer, referred to as the Ready-Exec sched-
uler, it calculates the set of threads R
exec
which are ready
to execute without directly blocking due to mutual exclu-
sion. That is, it examines whether an application thread will
try to enter a monitor which is already occupied by another
thread, if it is chosen as the next thread to execute. Having
calculated the set R
exec
, this layer passes it to the next layer.
The Safe-Exec scheduler layer is responsible for calcu-
lating the subset S
exec
of R
exec
, consisting of those threads
that can safely execute. Safety here refers both to deadlock
freedom (i.e., entering a monitor would not cause a dead-
lock later on), as well as, to meeting the timing constraints
of the different threads (i.e., choosing a thread for execu-
tion will not delay another thread enough to make it miss its
deadline).
The Safe-Exec layer passes the set S
exec
to the third layer
QoS-Exec, which calculates the set Q
exec
⊆ S
exec
, consist-
ing of the safe threads which respect the QoS requirements.
2.2. Controlling the Notified Thread
The right stack is passed control when the application
calls
notify
or
notifyAll
. The reason for this is that the
threads which will be notified (if any) cannot ever be se-
lected for execution. This is because they will immedi-
ately try to re-enter the monitor after being notified and thus
get blocked by the notifier (which is already in the moni-
tor). Nevertheless, when a thread notifies a condition vari-
able, then we can control which among the threads waiting
for the notification should receive the event. Indeed, lan-
guages (like JAVA [3]) which provide the monitor construct,
or thread libraries (like POSIX threads [6]) which offer it to
languages which do not provide it, leave this point unspec-
ified, allowing each implementation to choose the thread to
be notified as it convenes it the best.
The top layer Ready-Notif calculates the set R
notif
of
threads waiting on the condition variable being notified.
The apparent cases where we cannot effect any control
on the system are three. First, the case where no thread
waits on the condition variable being notified. Second, the
case where only one thread waits on the condition variable.
Third, the case where the notifying thread does a
notifyAll
,
in which case we are obliged to notify all the threads wait-
ing on the condition variable. In these cases, the right stack
of the scheduler does not make any control decision, but
simply changes the PC of all threads waiting on the current
notification, that is, the threads belonging to the set R
notif
,
to mark them as notified. Otherwise, the top layer passes
the set R
notif
to the middle layer Safe-Notif, which calcu-
lates the subset S
notif
of R
notif
consisting of those threads
which, if notified, will not cause the system to enter into a
deadlock state or cause some other thread to miss its dead-
line (i.e., they are safe). Finally, the Safe-Notif layer passes
the S
notif
set to the bottom QoS-Notif layer, which calcu-
lates the subset Q
notif
of S
notif
, consisting of the threads
which we can safely notify and also respect the QoS prop-
erties of the application. The QoS-Notif layer is also re-
sponsible for choosing one of the threads in Q
notif
as the
recipient of the current notification and marking it as noti-
fied by changing its PC.
2.3. QoS Policies
By incorporating a layer for QoS in the scheduler, we of-
fer an extendible mechanism for providing additional prop-
erties to the system. We allow the QoS policies to use the
same information which is available to the scheduler. That
is, the QoS layer has access to the program counters (PC) of
the threads, the currently executing thread (T
Exec
), as well
as the value of the system clock (C
System
).
The complexity of the QoS layer is controlled by the ap-
plication designer. In choosing a QoS policy (or policies,
since these are composable) the designer can balance be-
tween the execution time and extra memory space needed
by the policy and the gains to the overall system quality the
particular policy can offer. A QoS policy is, for example,
the local minimisation of context switches in order to speed-
up the execution. This policy can be implemented quite eas-
ily, since all one needs to examine is whether the currently
executing thread T
Exec
is in the set S
exec
of threads which
are safe to execute next. If this is the case, then we can let it
continue its execution, by setting the set Q
exec
equal to the
singleton {T
Exec
}. This particular policy has another ad-
R-T OS
Provide low-level mechanisms
Application
Application Tasks
QoS-Exec Scheduler
EXECUTING TASKS - Q
exec
⊆ S
exec
Ready-Exec Scheduler
Safe-Exec Scheduler
CANDIDATES FOR EXECUTION - C
exec
READY TASKS - R
exec
⊆ C
exec
SAFE TASKS - S
exec
⊆ R
exec
Execution Scheduler Stack
Mutual Exclusion Rules
Avoid Deadlocks
Guarantee Deadlines
Assure the QoS
QoS-Notif Scheduler
Notify tasks in Q
notif
Choose one among Q
exec
for execution
TASKS TO WAKE UP - Q
notif
⊆ S
notif
CANDIDATES FOR NOTIFICATIONS - C
notif
READY TO BE NOTIFIED - R
notif
⊆ C
notif
SAFE TARGETS - S
notif
⊆ R
notif
Safe-Notif Scheduler
Ready-Notif Scheduler
Notification Scheduler Stack
Assure the QoS
Guarantee Deadlines
Avoid Deadlocks
Condition Var. Notification Rules
Figure 1. A three-layered scheduler architecture
vantage: by decreasing the number of context switches, we
also decrease the cache misses of the application, since now
there are fewer points in the execution where the threads
compete for the cache, potentially flushing each other’s data
out of it. This can help decrease the energy consumption of
the system, since a cache miss can lead to two main memory
accesses, which are known to be quite demanding with re-
spect to energy [7]. In fact, since a cache reads and flushes
one cache line at a time (i.e., multiple consecutive mem-
ory addresses) the benefits can be even greater, both with
respect to energy consumption and execution speed.
3. System Model
The model of the system we construct is the parallel
composition of an automaton which is responsible for ad-
vancing time and firing the alarms, one automaton for each
of the application threads and two more automata, for the
QoS-Exec and the QoS-Notif scheduler layers respectively.
The automaton of time and the automata of the applica-
tion threads perform a finite number of actions and then
block, letting the scheduler automata respond. The actions
of the time and application automata being uncontrollable,
the only controllable actions are those of the two scheduler
automata. Thus, our model can be seen as a two player
game with the scheduler automata on one side (i.e., the con-
troller) and the time and application thread automata on the
other (i.e., the plant). In this game, the automata related
to the application simulate the locking and unlocking of
resources, as well as, the waiting and notification on con-
dition variables. The computations performed by the ap-
plication threads are simulated just by their minimum and
maximum execution times. Each statement s of an applica-
tion thread (where s is one of
monitorEnter
,
monitorExit
,
wait
,
waitTimed
,
waitForPeriod
, a conditional or a com-
putation) is modelled by a separate automaton state and a
transition from it to the next statement position (@s
0
) which
is taken when the statement s can be executed. The only
exception to this rule is the case of the
wait
and
waitTimed
statements. These statements are effectively modelled by
two states; the first one models the release of the mutex
associated with the condition variable on which we wait
and the second one models the attempt of the thread to re-
acquire the mutex, once it has been notified.The advance-
ment of time is the responsibility of a single automaton (see
Figure 2-a) which, in addition, enables transitions in the ap-
plication automata which correspond to timeouts, such as in
the case of a
waitTimed
or a
waitForPeriod
. This automa-
ton is also responsible for advancing the local thread clocks,
that is, the clocks which model the time spent by threads in
computations (and in waiting when doing a
waitTimed
).
These clocks are set to zero at the beginning of a compu-
tation by a thread and are incremented alongside with the
global time, until the duration of the computation is over
(or the timeout of the
waitTimed
has expired). The two
automata for the QoS-Exec and QoS-Notif scheduler layers
A
Alarm
alarm
=⇒ change state & pass control to the scheduler
¬ Alarm ∧ ¬ (Compute ∨ wait)
appli
=⇒ allow application to run
¬ Alarm ∧ (Compute ∨ wait)
tick
=⇒ advance clocks
2-a: Time automaton
t
i
∈ R
exec
∩ S
exec
∩ Q
exec
Choose t
i
=⇒ T
0
Exec
= t
i
A
2-b: QoS-Exec Scheduler automaton
Figure 2. Time & Scheduler automata
are passed control as described in sub-sections 2.1 and 2.2
and decide which of the application automata should be al-
lowed to execute next or be notified of an event. These au-
tomata are comprised of a single state and n + 1 transitions,
where n is the number of application threads (the additional
transition being for the idle thread). A transition of these
automata selects one of the threads for execution (resp. for
notification). It is guarded by a predicate which asserts that
the corresponding thread belongs to the Q
exec
(resp. Q
notif
)
set (see Figure 2-b). The transition choosing the idle thread
asserts that the rest of the threads are not safe to execute
(resp. no thread waits to be notified on the current event).
To summarise, the state in our model comprises of:(i) a
program counter (PC
i
) for each of the application threads,
(ii) a local clock (C
i
) for each thread which is used for their
computations and the timeouts if they execute a
waitTimed
,
(iii) a global clock (C
System
i
) for modelling the periods
of each periodic thread, (iv) a variable (T
Exec
) holding
the currently executing thread, (v) two boolean variables
(Exec Sched Enabled & Notif Sched Enabled) for control-
ling whether it is one of the scheduler automata (and which
of them), or the time (when they are both true) or the
time and application automata (when they are both false)
which should execute, and (vi) the boolean variables of
the application threads used in conditionals which are sig-
nificant
1
with respect to the use of resources and com-
munication of events. Our model goes through three dif-
ferent modes of execution, as shown in Figure 3. In
the “Time Only” mode (where Exec Sched Enabled =
Notif Sched Enabled = true) the time automaton is the
sole automaton enabled in the system and it can fire one
or more alarms, if any is enabled. If an alarm is fired then
the execution mode changes to “Schedulers Only” (where
1
We perform slicing of the original code to identify these variables.
New tick, so check alarms
No Alarms ⇒ Allow Application to run
Re-Schedule
Alarm(s) fired
Check alarms
Schedulers
Only
Time
Only
Time
and
Application
Figure 3. Model execution modes
Exec Sched Enabled = ¬Notif Sched Enabled), so that our
scheduler can treat the alarm. If there is no alarm to be fired
then the execution mode changes to “Time and Application”
(where Exec Sched Enabled = Notif Sched Enabled =
false). At this mode, both the time automaton and the au-
tomata of the application are enabled. If the time automaton
gets to execute first, then a tick (i.e., a time step) is per-
formed and we pass back to the “Time Only” mode, so as
to check if an alarm is now enabled. If it is one of the ap-
plication automata which gets to execute first, then it exe-
cutes until it needs to perform an action which causes re-
scheduling, in which case it passes control to the schedulers
(i.e., the mode now becomes “Schedulers Only”). If the
application automaton needs to execute a time guarded ac-
tion (i.e., a computation), then it blocks, allowing time to
advance.
As an example, let us consider the model shown in Fig-
ure 4. Here, the application consists of three threads, one of
which is a periodic one (the
User
) and two aperiodic ones
(the
Writer
and the
Refresher
). One should note that
the
Writer
and the
Refresher
are continually enabled
aperiodic threads and do not have any deadlines directly as-
sociated with them.
4. Scheduler Synthesis
In order to synthesise the Safe-Exec and Safe-Notif
scheduler layers, we first construct the set of reachable
states and, thus, identify the deadlocks. These are the states
where the application threads are deadlocked, or the states
where some thread has missed its deadline (since in that
case we block the system explicitly). The existence of these
states indicates that the predicate we are currently using to
describe the set Safe-Exec (resp. Safe-Notif ) needs to be
constrained even further. This predicate starts with the value
of
true
, thus accepting initially all threads in the set Ready-
Exec (resp. Ready-Notif) as safe. Having obtained the dead-
locked states, we do a backwards traversal of the whole state
space starting from the deadlocked states, until we reach a
