HAL Id: hal-01413212
https://hal.inria.fr/hal-01413212
Submitted on 12 Dec 2016
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Load-aware shedding in stream processing systems
Nicoló Rivetti, Yann Busnel, Leonardo Querzoni
To cite this version:
Nicoló Rivetti, Yann Busnel, Leonardo Querzoni. Load-aware shedding in stream processing systems.
10th ACM International Conference on Distributed and Event-based Systems (DEBS), Jun 2016,
Irvine, United States. pp.61 - 68, �10.1145/2933267.2933311�. �hal-01413212�
Load-Aware Shedding in Stream Processing Systems
Nicoló Rivetti
LINA / Université de Nantes,
France
DIAG / Sapienza University of
Rome, Italy
rivetti@dis.uniroma1.it
Yann Busnel
Crest (Ensai) / Inria
Rennes, France
yann.busnel@ensai.fr
Leonardo Querzoni
DIAG / Sapienza University of
Rome, Italy
querzoni@dis.uniroma1.it
ABSTRACT
Load shedding is a technique employed by stream process-
ing systems to handle unpredictable spikes in the input load
whenever available computing resources are not adequately
provisioned. A load shedder drops tuples to keep the input
load below a critical threshold and thus avoid tuple queuing
and system trashing. In this paper we propose Load-Aware
Shedding (LAS), a novel load shedding solution that drops
tuples with the aim of maintaining queuing times below a
tunable threshold. Tuple execution durations are estimated
at runtime using efficient sketch data structures. We pro-
vide a theoretical analysis proving that LAS is an (ε, δ)-
approximation of the optimal online load shedder and show
its performance through a practical evaluation based both
on simulations and on a running prototype.
CCS Concepts
•Software and its engineering → Distributed systems
organizing principles; •Theory of computation → On-
line learning algorithms; Sketching and sampling;
Keywords
Stream Processing, Data Streaming, Load Shedding
1. INTRODUCTION
Distributed stream processing systems (DSPS) are today
considered as a mainstream technology to build architec-
tures for the real-time analysis of big data. An application
running in a DSPS is typically modeled as a directed acyclic
graph where data operators (nodes) are interconnected by
streams of tuples containing data to be analyzed (edges).
The success of such systems can be traced back to their
ability to run complex applications at scale on clusters of
commodity hardware.
Correctly provisioning computing resources for DSPS how-
ever is far from being a trivial task. System designers need
Accepted to the 10th ACM International Conference on Distributed and Event-based
Systems (DEBS ’16)
to take into account several factors: the computational com-
plexity of the operators, the overhead induced by the frame-
work, and the characteristics of the input streams. This
latter aspect is often critical, as input data streams may
unpredictably change over time both in rate and content.
Bursty input load represents a problem for DSPS as it may
create unpredictable bottlenecks within the system that lead
to an increase in queuing latencies, pushing the system in a
state where it cannot deliver the expected quality of service
(typically expressed in terms of tuple completion latency).
Load shedding is generally considered a practical approach
to handle bursty traffic. It consists in dropping a subset of
incoming tuples as soon as a bottleneck is detected in the
system.
Existing load shedding solutions either randomly drop tu-
ples when bottlenecks are detected or apply a pre-defined
model of the application and its input that allows them to
deterministically take the best shedding decision. In any
case, all the existing solutions assume that incoming tuples
all impose the same computational load on the DSPS. How-
ever, such assumption does not hold for many practical use
cases. The tuple execution duration, in fact, may depend on
the tuple content itself. This is often the case whenever the
receiving operator implements a logic with branches where
only a subset of the incoming tuples travels through each sin-
gle branch. If the computation associated with each branch
generates different loads, then the execution duration will
change from tuple to tuple. A tuple with a large execution
duration may delay the execution of subsequent tuples in the
same stream, thus increasing queuing latencies and possibly
cause the emergence of a bottleneck.
On the basis of this simple observation, we introduce Load-
Aware Shedding (LAS), a novel solution for load shedding in
DSPS. LAS gets rid of the aforementioned assumptions and
provides efficient shedding aimed at matching given queuing
latency goals, while dropping as few tuples as possible. To
reach this goal LAS leverages a smart combination of sketch
data structures to efficiently collect at runtime information
on the time needed to compute tuples and thus build and
maintain a cost model that is then exploited to take deci-
sions on when load must be shed. LAS has been designed as
a flexible solution that can be applied on a per-operator ba-
sis, thus allowing developers to target specific critical stream
paths in their applications.
In summary, the contributions provided by this paper are
(i) the introduction of LAS, the first solution for load shed-
ding in DSPS that proactively drops tuples to avoid bottle-
necks without requiring a predefined cost model and without
any assumption on the distribution of tuples, (ii) a theo-
retical analysis of LAS that points out how it is an (, δ)-
approximation of the optimal online shedding algorithm and,
finally, (iii) an experimental evaluation that illustrates how
LAS can provide predictable queuing latencies that approx-
imate a given threshold while dropping a small fraction of
the incoming tuples.
Below, the next section states the system model we con-
sider. Afterwards, Section 3 details LAS whose behavior is
then theoretically analyzed in Section 4. Section 5 reports
on our experimental evaluation and Section 6 analyzes the
related works. Finally Section 7 concludes the paper.
2. SYSTEM MODEL AND PROBLEM DEF-
INITION
We consider a distributed stream processing system (DSPS)
deployed on a cluster where several computing nodes ex-
change data through messages sent over a network. The
DSPS executes a stream processing application represented
by a topology: a directed acyclic graph interconnecting op-
erators, represented by vertices, with data streams (DS),
represented by edges.
Data injected by source operators is encapsulated in units
called tuples and each data stream is an unbounded sequence
of tuples. Without loss of generality, here we assume that
each tuple t is a finite set of key/value pairs that can be
customized to represent complex data structures. To sim-
plify the discussion, in the rest of this work we deal with
streams of unary tuples each representing a single non nega-
tive integer value. We also restrict our model to a topology
with an operator LS (load shedder) that decides which tu-
ples of its outbound DS σ consumed by operator O shall
be dropped. Tuples in σ are drawn from a large universe
[n] = {1, . . . , n} and are ordered, i.e., σ = ht
1
, . . . , t
m
i.
Therefore [m] = 1, . . . , m is the index sequence associated
with the m tuples contained in the stream σ. Both m and
n are unknown. We denote with f
t
the unknown frequency
of tuple t, i.e., the number of occurrences of t in σ.
We assume that the execution duration of tuple t on oper-
ator O, denoted as w(t), depends on the content of t. In par-
ticular, without loss of generality, we consider a case where
w depends on a single, fixed and known attribute value of
t. The probability distribution of such attribute values, as
well as w, are unknown, may differ from operator to oper-
ator and may change over time. However, we assume that
subsequent changes are interleaved by a large enough time
frame such that an algorithm may have a reasonable amount
of time to adapt. On the other hand, the input throughput
of the stream may vary, even with a large magnitude, at any
time.
Let q(i) be the queuing latency of the i-th tuple of the
stream, i.e., the time spent by the i-th tuple in the inbound
buffer of operator O before being processed. Let us denote
as D ⊆ [m], the set of dropped tuples in a stream of length
m, i.e., dropped tuples are thus represented in D by their
indices in the stream [m]. Moreover, let d ≤ m be the
number of dropped tuples in a stream of length m, i.e.,
d = |D|. We can define the average queuing latency as:
Q(j) =
P
i∈[j]\D
q(i)/(j − d) for all j ∈ [m].
The goal of the load shedder is to maintain the average
queuing latency smaller than a given threshold τ by drop-
ping as less tuples as possible while the stream unfolds. The
quality of the shedder can be evaluated both by comparing
the resulting Q against τ and by measuring the number of
dropped tuples d. More formally, the load shedding problem
can be defined as follows
1
:
Problem 2.1 (Load Shedding). Given a data stream
σ = ht
1
, . . . , t
m
i, find the smallest set D such that
∀j ∈ [m] \ D, Q(j) ≤ τ.
3. LOAD AWARE SHEDDING
3.1 Overview
Load-Aware Shedding (LAS) is based on a simple, yet
effective, idea: if we assume to know the execution duration
w(t) of each tuple t in the operator, then we can foresee
queuing times and drop all tuples that will cause the queuing
latency threshold τ to be violated. However, the value of
w(t) is generally unknown.
LAS builds and maintain at run-time a cost model for tu-
ple execution durations. It takes shedding decision based on
the estimation
b
C of the total execution duration of the oper-
ator: C =
P
i∈[m]\D
w(t
i
). In order to do so, LAS computes
an estimation ˆw(t) of the execution duration w(t) of each
tuple t. Then, it computes the sum of the estimated exe-
cution durations of the tuples assigned to the operator, i.e.,
b
C =
P
i∈[m]\D
ˆw(t). At the arrival of the i-th tuple, subtract-
ing from
b
C the (physical) time elapsed from the emission of
the first tuple provides LAS with an estimation ˆq(i) of the
queuing latency q(i) for the current tuple.
To enable this approach, LAS builds a sketch on the oper-
ator (i.e., a memory efficient data structure) that will track
the execution duration of the tuples it processes. When
a change in the stream or operator characteristics affects
the tuples execution durations w(t), i.e., the sketch con-
tent changes, the operator will forward an updated version
to the load shedder, which will than be able to (again) cor-
rectly estimate the tuples execution durations. This solution
does not require any a priori knowledge on the stream or
system, and is designed to continuously adapt to changes in
the input stream or on the operator characteristics.
3.2 LAS design
The operator maintains two Count Min [4] sketch matrices
(Figure 1.A): the first one, denoted as F, tracks the tuple
frequencies f
t
; the second one, denoted as W, tracks the tu-
ples cumulated execution durations W
t
= w(t) × f
t
. Both
Count Min matrices share the same sizes and 2-universal
hash functions [3]. The latter is a generalized version of the
Count Min providing (ε, δ)-additive-approximation of point
queries on stream of updates whose value is the tuple execu-
tion duration when processed by the instance. The operator
will update (Listing 3.1 lines 27-30) both matrices after each
tuple execution.
The operator is modeled as a finite state machine (Fig-
ure 2) with two states: START and STABILIZING. The
START state lasts as long as the operator has executed N
tuples, where N is a user defined window size parameter.
The transition to the STABILIZING state (Figure 2.A)
1
This is not the only possible definition of the load shedding
problems. Other variants are briefly discussed in section 6.
c
1 2 3 4
r
2
1
F
c
1 2 3 4
W
hF, Wi
O
LAS
b
C
hF, Wi
LS
htuplei | htuple,
b
Ci
hF, Wi
h∆i
A
B
C
D
E
Figure 1: Load-Aware Shedding design with r = 2
(δ = 0.25), c = 4 (ε = 0.70).
triggers the creation of a new snapshot S. A snapshot is
a matrix of size r × c where ∀i ∈ [r], j ∈ [c] : S[i, j] =
W[i, j]/F[i, j] (Listing 3.1 lines 15-17). We say that the F
and W matrices are stable when the relative error η between
the previous snapshot and the current one is smaller than a
configurable parameter µ, i.e.,
η =
P
∀i,j
|S[i, j] −
W[i,j]
F[i,j])
|
P
∀i,j
S[i, j]
≤ µ (1)
is satisfied. Then, each time the operator has executed N tu-
ples (Listing 3.1 lines 18-25), it checks whether Equation 1 is
satisfied. (i) In the negative case S is updated (Figure 2.B).
(ii) In the positive case the operator sends the F and W ma-
trices to the load shedder (Figure 1.B), resets their content
and moves back to the START state (Figure 2.C).
There is a delay between any change in w(t) and when LS
receives the updated F and W matrices. This introduces a
skew in the cumulated execution duration estimated by LS.
In order to compensate this skew, we introduce a synchro-
nization mechanism that kicks in whenever the LS receives
a new pair of matrices from the operator.
The LS (Figure 1.C) maintains the estimated cumulated
execution duration of the operator
b
C and a pairs of initially
empty matrices hF, Wi. LS is modeled as a finite state ma-
chine (Figure 3) with three states: NOP, SEND and RUN.
The LS executes the code reported in Listing 3.2. In partic-
ular, every time a new tuple t arrives at the LS, the function
shed is executed. The LS starts in the NOP state where
no action is performed (Listing 3.2 lines 15-17). Here we
assume that in this initial phase, i.e., when the topology
has just been deployed, no load shedding is required. When
LS receives the first pair hF, Wi of matrices (Figure 3.A),
it moves into the SEND state and updates its local pair of
matrices (Listing 3.2 lines 7-9). While being in the SEND
states, LS sends to O the current cumulated execution du-
ration estimation
b
C (Figure 1.D) piggy backing it with the
first tuple t that is not dropped (Listing 3.2 lines 24-26) and
moves in the RUN state (Figure 3.B). This informations is
used to synchronize the LS with O and remove the skew
between O’s cumulated execution duration C and the esti-
mation
b
C at LS. O replies to this request (Figure 1.E) with
the difference ∆ = C −
b
C (Listing 3.1 lines 11-13). When the
load shedder receives the synchronization reply (Figure 3.C)
it updates its estimation
b
C + ∆ (Listing 3.2 lines 11-13).
Listing 3.1: Operator
1: init do
2: F ← 0
r,c
zero matrices of size r × c
3: W ← 0
r,c
4: S ← 0
r,c
5: r hash functions h
1
, . . . , h
r
: [n] → [c] from a 2-universal
family.
6: m ← 0
7: state ← Start
8: end init
9: function update(tuple : t, execution time : l, request :
b
C)
10: m ← m + 1
11: if
b
C not null then
12: ∆ ← C −
b
C
13: send h∆i to LS
14: end if
15: if state = Start ∧ m mod N = 0 then Figure 2.A
16: update S
17: state ← Stabilizing
18: else if state = Stabilizing ∧ m mod N = 0 then
19: if η ≤ µ (Eq. 1) then Figure 2.C
20: send hF, Wi to LS
21: state ← Start
22: reset F and W to 0
r,c
23: else Figure 2.B
24: update S
25: end if
26: end if
27: for i = 1 to r do
28: F[i, h
i
(t)] ← F[i, h
i
(t)] + 1
29: W[i, h
i
(t)] ← W[i, h
i
(t)] + l
30: end for
31: end function
start stabilizing
execute N tuples
create snapshot S
execute N tuples ∧ relative error η ≤ µ
send F and W to scheduler and reset them
execute N tuples ∧
relative error η > µ
update snapshot S
A
B
C
Figure 2: Operator finite state machine.
In the RUN state, the load shedder computes, for each
tuple t, the estimated queuing latency ˆq(i) as the difference
between the operator estimated execution duration
b
C and
the time elapsed from the emission of the first tuple (List-
ing 3.2 line 18). It then checks if the estimated queuing
latency for t satisfies the Check method (Listing 3.2 lines
19-21).
This method encapsulates the logic for checking if a de-
sired condition on queuing latencies is violated or not. In
this paper, as stated in Section 2, we aim at maintaining the
average queuing latency below a threshold τ. Then, Check
tries to add ˆq to the current average queuing latency (List-
ing 3.2 lines 31). If the result is larger than τ (i), it simply
returns true; otherwise (ii), it updates its local value for
the average queuing latency and returns f alse (Listing 3.2
lines 34-36). Note that different goals, based on the queuing
latency, can be defined and encapsulated within Check.
If Check(ˆq) returns true, (i) the load shedder returns
true as well, i.e., tuple t must be dropped. Otherwise (ii),
the operator estimated execution duration
b
C is updated with
Send RUN
NOP
synhcronization request
sent
received reply
resynchronize
b
C
received F and W
update local F and W
A
B
C
D
Figure 3: Load shedder LS finite state machine.
the estimated tuple execution duration ˆw(t), increased by a
factor 1 + ε to mitigate potential under-estimations
2
, and
the load shedder returns false (Listing 3.2 line 28), i.e.,
the tuple must not be dropped. Finally, if the load shed-
der receives a new pair hF, Wi of matrices (Figure 3.D), it
will again update its local pair of matrices and move to the
SEND state (Listing 3.2 lines 7-9).
Now we will briefly discuss the complexity of LAS.
Theorem 3.1 (Time complexity of LAS).
For each tuple read from the input stream, the time complex-
ity of LAS for the operator and the load shedder is O(log 1/δ).
Theorem 3.2 (Space Complexity of LAS).
The space complexity of LAS for the operator and load shed-
der is O
1
ε
log
1
δ
(log m + log n)
bits.
Theorem 3.3 (Communication complexity of LAS).
The communication complexity of LAS is of O
m
N
messages
and O
m
N
1
ε
log
1
δ
(log m + log n) + log m
bits.
Note that the communication cost is low with respect to
the stream size since the window size N should be chosen
such that N 1 (e.g., in our tests we have N = 1024).
Proofs for the previous theorems are available in [8].
4. THEORETICAL ANALYSIS
Data streaming algorithms strongly rely on pseudo-random
functions that map elements of the stream to uniformly dis-
tributed image values to keep the essential information of the
input stream, regardless of the stream elements frequency
distribution. Here we provide the analysis of the quality
of the shedding performed by LAS in two steps. First we
study the correctness and optimality of the shedding algo-
rithm, under full knowledge assumption (i.e., the shedding
strategy is aware of the exact execution duration w
t
for each
tuple t). For the sake of clarity, and to abide to space limits,
all the proofs are available in a companion paper [8].
We suppose that tuples cannot be preempted, i.e. they
must be processed in an uninterrupted fashion on the avail-
able operator instance. Given our system model, here we
consider the problem of minimizing d, the number of dropped
tuples, while guaranteeing that the average queuing latency
Q(t) will be upper-bounded by τ, ∀t ∈ σ. The solution must
work online, thus the decision of enqueueing or dropping a
2
This correction factor derives from the fact that ˆw(t) is a
(ε, δ)-approximation of w(t) as shown in Section 4.
Listing 3.2: Load shedder
1: init do
2:
b
C ← 0
3: hF, Wi ← h0
r,c
, 0
r,c
i zero matrices pair of size r × c
4: Same hash functions h
1
. . . h
r
of the operator
5: state ← NOP
6: end init
7: upon hF
0
, W
0
i do Figure 3.A and 3.D
8: state ← Send
9: hF, Wi ← hF
0
, W
0
i
10: end upon
11: upon h∆i do Figure 3.C
12:
b
C ←
b
C + ∆
13: end upon
14: function shed(tuple : t)
15: if state = NOP then
16: return false
17: end if
18: ˆq ←
b
C− elapsed time from first tuple
19: if Check(ˆq) then
20: return true
21: end if
22: i ← arg min
i∈[r]
{F[i, h
i
(t)]}
23:
b
C ←
b
C + (W[i, h
i
(t)]/F[i, h
i
(t)]) × (1 + ε)
24: if state = Send then Figure 3.B
25: piggy back
b
C to operator on t
26: state ← Run
27: end if
28: return false
29: end function
30: function Check(q)
31: if Q/ > τ then
32: return true
33: end if
34: Q ← Q + q
35: ← + 1
36: return false
37: end function
tuple has to be made only resorting to knowledge about tu-
ples received so far in the stream.
Let OPT be the online algorithm that provides the opti-
mal solution to Problem 2.1. We denote with D
σ
OP T
(resp.
d
σ
OP T
) the set of dropped tuple indices (resp. the number
of dropped tuples) produced by the OPT algorithm fed by
stream σ (cf., Section 2). We also denote with d
σ
LAS
the
number of dropped tuples produced by LAS introduced in
Section 3.2 fed with the same stream σ.
Theorem 4.1 (LAS Correctness & Optimality).
For any σ, we have d
σ
LAS
= d
σ
OP T
and ∀t ∈ σ,
Q
σ
LAS
(t) ≤ τ.
Now we remove the previous assumptions and analyze the
approximation made on execution duration w(t) for each
tuple t. LAS uses two matrices, F and W, to estimate the
execution time w(t) of each tuple submitted to the operator.
By the Count Min sketch algorithm [4] and Listing 3.1, we
have that for any t ∈ [n] and for each row i ∈ [r],
F[i][h
i
(t)] = f
t
+
n
X
u=1,u6=t
f
u
1
{h
i
(u)=h
i
(t)}
.
and
W[i][h
i
(t)] = f
t
w(t) +
n
X
u=1,u6=t
f
u
w(u)1
{h
i
(u)=h
i
(t)}
,