HAL Id: hal-02515937
https://hal.archives-ouvertes.fr/hal-02515937
Submitted on 23 Mar 2020
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Online shortest paths with condence intervals for
routing in a time varying random network
Stephane Chretien, Christophe Guyeux
To cite this version:
Stephane Chretien, Christophe Guyeux. Online shortest paths with condence intervals for routing in
a time varying random network. 2018 International Joint Conference on Neural Networks (IJCNN),
2018, Rio, Brazil. �hal-02515937�
Online shortest paths with confidence intervals for
routing in a time varying random network
St
´
ephane Chr
´
etien
London Physical Laboratory, UK
Email: stephane.chretien@npl.co.uk
Christophe Guyeux
Institut Femto-ST, UMR 6174 CNRS
Universit
´
e de Bourgogne Franche-Comt
´
e, France
Email: christophe.guyeux@femto.fr
Abstract—The increase in the world’s population and rising
standards of living is leading to an ever-increasing number of
vehicles on the roads, and with it ever-increasing difficulties
in traffic management. This traffic management in transport
networks can be clearly optimized by using information and
communication technologies referred as Intelligent Transport
Systems (ITS). This management problem is usually reformulated
as finding the shortest path in a time varying random graph. In
this article, an online shortest path computation using stochastic
gradient descent is proposed. This routing algorithm for ITS
traffic management is based on the online Frank-Wolfe approach.
Our improvement enables to find a confidence interval for the
shortest path, by using the stochastic gradient algorithm for ap-
proximate Bayesian inference. The theory required to understand
our approach is provided, as well as the implementation details.
I. INTRODUCTION
A. Motivation
Information and communication technologies have a great
potential impact in the field of traffic and mobility manage-
ment with a goal to achieve a safer and better optimized use
of the transport network [7]. Optimal routing is one of the key
tasks involved in intelligent planning and an extensive body
of work has been devoted to this topic since the 60’s. The
optimal routing problem is well known in the combinatorial
optimisation community under the name of the shortest path
problem. Since the advent of Linear Programming in the 50’s,
a great number of optimisation problems related to logistics
and planning have be shown to be special instances of convex
optimisation. The shortest path problem, in particular, is one
of these problems for which a solution can be found in
polynomial time using linear programming, and even faster
algorithms are available such as the celebrated Dijkstra algo-
rithm (or its generalisation, namely the A* algorithm).
In real world situations, however, there is never such a thing
as a static network and the data available on the edge properties
are evolving drastically in time. The cause of these variations
are most often multiple: in e.g. transportation networks, a
periodic behavior is definitely present in the traversal time
of every route segment of the graph, because of the signal
repeats from one day to the other. There might also be other
periodicities such as depending on the day in the week,
on the season, etc. There must also be various types of
trends: linear or piecewise linear trends, jumps, exponential
growth, etc. Jumps may be due to various kinds of disruptions.
Piecewise linear signals can also occur in the presence of
certain saturation phenomena. Exponential growth may be due
to the increase of the population. On the top of this list of
possible variations of the network properties come the fully
random fluctuations, due to real time behavior on the roads.
B. Previous work on the stochastic shortest path problem
In the case where the network is corrupted by random
perturbations only, a very important work appeared in [3]
where the problem is discussed in great mathematical details
using a Markovian decision process approach. This model was
also explored in [15]. The dynamic programming approach
was later refined in the subsequent paper [14] where the
computational complexity is significantly improved. The paper
[18] described a model with possible spatial and temporal
correlations between the edges of the network. Starting from
the results in [18], the interesting work [8] described a efficient
algorithm for solving the stochastic shortest path problem with
correlations. A non-stationary version of this kind of model
was proposed in [9] where the problem is addressed using an
adaptation of dynamic programming ideas.
Our approach to the shortest path problem was inspired by
the recent breakthrough of [1] where the problem was recast
as an online convex optimisation problem and the rate m
3
2
√
T
was proved for a practically easy to implement algorithm. The
online approach to the stochastic shortest path problem had
long been interesting to the machine learning commmunity.
The pioneering work by [17] and then [12] set on the trend
and proposed interesting and efficient algorithm. The partial
information setting was studied in [2] and superseeded by
[1] using online convex optimisation techniques based on
the linear programming representation of the shortest path
problem based on the handy convex hull representation of
P
u,v
.
C. Our contribution
The previous works on the stochastic shortest path cul-
minated in the discovery of the adaptive approach of [1]
for the bandit setting with a controlled regret. A recent and
elementary presentation of the online Frank-Wolfe approach
to the problem is proposed in [11, Section 7]. Our interest in
the online approach was sparked by the difficulty to model
the intricate compound of random and deterministic behavior
978-1-5090-6014-6/18/$31.00 ©2018 IEEE
which enter into the type of phenomena observable on real
networks.
The main ingredient we contribute to the online approach
is a way to produce a confidence interval together with a
shortest path. Our procedure is motivated by recent discoveries
[13] which prove that an appropriate rescaling of the Iterate
Averaging Stochastic Gradient produces a sequence which
is nearly Gaussian and has a distribution governed by the
Bayesian posterior associated with the estimation problem, i.e.
the shortest path in our setting.
The main algorithm proposed in this paper combines the
results of [11] with those of [13] in order to propose an
efficient way to simulate the distribution at time T of the
shortest path in a stochastic and potentially non-stationary (but
smoothly evolving) environment.
We demonstrate through numerical experiments the effi-
ciency of our approach.
D. Plan of the paper
The remainder of this article is organized as follows. The
proposed algorithm is described in Section II together with its
practical implementation. Numerical simulations are then pro-
posed in Section III, for the sake of illustration. This research
work ends by a conclusion section, in which the contribution
is summarized and intended future work is outlined.
II. ONLINE ROUTING
In this section, we describe our algorithm and its practical
implementation.
A. Online shortest path computation using stochastic gradient
descent
The shortest path problem in a time varying random graph
is more complex to address than the standard shortest path
problem in a deterministic graph.
1) Formulation: We are given a graph G = (V, E) and
a source-sink pair (u, v) ∈ V
2
. Let P
u,v
denote the set of
all paths from u to v. At each iteration t, the optimisation
algorithm observes a set of weights w
t
∈ R
|E|
+
and selects a
path p
t
∈ P
u,v
. When the next iteration t + 1 starts, a new set
of observed weights w
t+1
∈ R
|E|
+
is available and the decision
maker observes a loss
l
t
=
X
e∈p
t
w
t
(e) (1)
associated with p
t
. Our goal is to design a stochastic algorithm
in order to perform iterative routing optimisation based on
online sequentially updated information w
t
about the network
state.
One possible way to address this stochastic optimisation
problem is to consider each path in P
u,v
as an expert and
compare the loss of all experts in the pool in order to select
the best among them. Unfortunately, the cardinality of P
u,v
might be exponential and the resulting optimisation problem
might end up being computationally intractable.
Another, much more efficient option is to take the online
convex optimisation approach of [11].
2) The online Frank-Wolfe algorithm: As is well known in
combinatorial optimisation, the standard shortest path problem
with edge weights w
e
, e ∈ E, can be recast as a linear
programming problem consisting in solving
min
x∈E
|E|
w
∗
x (2)
subject to
1
x ∈ P
u,v
. The main idea is based on the fact that
the P
u,v
has the following convex hull:
X
c∈E
x
u,c
= 1 (3)
X
c∈E
x
c,v
= 1 (4)
∀c ∈ V \ {u, v},
X
c
0
:(c
0
,c)∈E
x
c
0
,c
=
X
c
0
:(c,c
0
)∈E
x
c,c
0
(5)
∀e ∈ E, x
e
∈ [0, 1]. (6)
In our time varying random setting, we receive at each time
t a new value w
t
∈ R
|E|
of the vector of weights and our goal
is to achieve the minimal regret over a given time horizon
{1, . . . , T }, for a given T ∈ N
∗
. The Online Conditional
Gradient, aka Online Frank-Wolfe algorithm is a very efficient
approach for doing this. The method is described in Algorithm
1 below.
Result: The path encoded in x
T
x
1
is a shortest path from u to v, computed e.g. from
averaged historical data;
for t = 1 to T do
y
t+1
← shortest path with edge cost vector
η
t
X
τ=1
w
τ
+ 2(x
t
− x
1
) (7)
set x
t+1
= γ
t
x
t
+ (1 − γ
t
) y
t+1
;
end
Algorithm 1: The Online Frank-Wolfe Algorithm [11, Sec-
tion 7.5]
The following result characterises the performance and
adaptivity of this online method.
Theorem 1 [11, Theorem 7.2] Let D be the diameter of
conv(P
u,v
) and assume that kw
t
k
2
≤ G for all t ∈ {1, . . . , T }.
The Online Frank-Wolfe Algorithm 1 with parameters η =
G/(DT
3/4
) and γ
t
= min{1, 2/t
1/2
} satisfies
∗
X
t=1
w
∗
t
x
t
− min
x∈conv(P
u,v
)
T
X
t=1
w
∗
t
x ≤ 8DG T
3/4
. (8)
In other words, after dividing (8) by T , this theorem says that
the average loss suffered by the sequence of shortest paths is
of the same order as the best average loss, i.e. the one suffered
by one optimal path.
1
here we make a slight abuse of notation by interpreting P
u,v
as a set of
binary vectors which are indicators of a path from u to v, i.e. setting every
component of x to 1 when index by an edge on the path
2018 International Joint Conference on Neural Networks (IJCNN)
B. Joint posterior distribution and confidence intervals
The next question to be addressed is the one of providing
a confidence interval for the shortest path. One idea is to
apply the approach proposed in [13] which consists in using
the stochastic gradient algorithm for approximate Bayesian
Inference.
In the case where, instead of the Online Frank-Wolfe
algorithm, one uses the Iterate Averaging Stochastic Gradient
method defined in Algorithm 2. For this algorithm to be
Result: The last iterate x
T
x
1
is an arbitrary initial point;
for t = 1 to T do
y
t+1
= y
t
− η∇f
t
(x
t
)
set x
t+1
= γ
t
x
t
+ (1 − γ
t
) y
t+1
;
end
Algorithm 2: The Iterate Averaging Stochastic Gradient
Algorithm [13, Section6.1]
relevant, however, we need the function f to be strongly
convex. Moreover, the vector ∇f
t
(x
t
) will denote a centered
stochastic perturbation of ∇f(x
t
).
For the Iterate Averaging Stochastic Gradient Algorithm,
one obtains the following guarantees.
Claim 1 [13, Based on Appendix G] Assume that
L
T
(x) =
1
T
T
X
t=1
f
t
(x) (9)
is strongly convex. Take
γ
t
=
t
t + 1
. (10)
Then, we have
E [x
T
x
∗
T
] ≈
1
ηT
Σ(A
−1
)
∗
+ A
−1
Σ
(11)
where A is the Hessian of L
T
at its minimiser.
The accuracy of ≈ in the latter claim depends on how well L
T
is approximated by
1
2
x
t
Ax on the domain where the trajectory
(x
t
)
t=1,...,T
will be wandering.
C. Our new approach
In our Online Frank-Wolfe setting, the problem is radi-
cally different from the Iterate Averaging Stochastic Gradient
method, since the average cost L
T
=
1
T
P
T
t=1
hw
t
, ·i is linear,
and therefore not strongly convex. However, we can use the
same type of result for the output x
T
generated by the Online
Frank-Wolfe Algorithm with γ
t
= t/(t + 1) as in (10), and
η = 1/T . Indeed, a similar diffusion-type approximation can
be performed for the Online Frank Wolfe Algorithm as the one
given in [13] for the Iterate Averaging Stochastic Gradient if
we replace the Ornstein-Uhlenbeck process by one similar to
[6]. The theoretical details will be given in an extended version
of the present work.
Result: The last iterate x
T
for l = 1 to L do
Run Algorithm 1 with γ
t
= t/(t + 1) and η = 1/T
and return x
(l)
T
end
Compute the variance
ˆσ =
1
L
L
X
l=1
w
(l)∗
T
x
(l)
T
2
−
1
L
L
X
l=1
w
(l)∗
T
x
(l)
T
!
2
. (12)
Output the confidence interval
I
α
=
"
1
L
L
X
l=1
w
(l)∗
T
x
(l)
T
− u
1−α/2
;
1
L
L
X
l=1
w
(l)∗
T
x
(l)
T
+ u
1−α/2
#
(13)
Algorithm 3: The new sampling scheme
III. NUMERICAL SIMULATION
A. Simulation protocol
We have designed an ad hoc transport network using the
Python language [16]. In this simulator, the number of cross-
roads has been set at various values, and each of them is
linked to 2 to 5 other crossroads, according to a random draw.
This design leads to a undirected graph, where nodes represent
the crossroads, and edges are weighted according to the time
needed between two nodes. Such weights are randomly picked
in the real interval [0; 1] and updated at each time iterate,
leading to a time varying random network. Such a dynamic
graph has been implemented using the Networkx library [10].
Algorithm 1 has then been computed, with the following
parameters. η has been set to:
η =
G
DT
3
2
,
where G is equal to the square root of the number of edges,
while D is the largest distance between two shortest paths. T ,
for its part, is the number of steps of the algorithm, which has
been set at 100. Finally, we have considered that:
γ
t
=
t
t + 1
.
At initialization stage, w
0
, which is the vector of travel
times (from size the number of edges) at the beginning of the
simulation, which should in practice be equal to the averaged
historical data, has been picked randomly. y
1
is the associated
shortest path calculated, like all the other shortest paths, using
the Dijkstra algorithm: the i-th component of y
1
is 1 if the
i-th edge is in this shortest path, else it is equal to 0. Finally,
x
1
has been set at y
1
.
B. Obtained results
We have applied the proposed algorithm to various random
networks, with a number of nodes respectively equal to
N = 12, 100, 200, and 500. The evolution of the shortest
2018 International Joint Conference on Neural Networks (IJCNN)
(a) t=1 (b) t=5
(c) t=10 (d) t=15
(e) t=20
Fig. 1: The algorithm working on a small dynamic network (reaching 11 from 0)
paths between any couple of nodes has been stored as movies,
in which cumulative times between two locations are put on
edges, while at each time only edges of the shortest path
are not dotted. Examples of obtained results are depicted in
Figure 1 for 5 iteration steps and 12 nodes, while the mid term
evolution of the algorithm to the shortest paths is presented
in Figure 2 for 10 couples of nodes within a network of 200
vertices.
Finally, we have verified that the global computation cost of
our approach is similar to the Djikstra algorithm (complexity
of O(nlog(n)), as can be seen in Figure 3. This result is
encouraging, as we take place in a more difficult context of
the shortest path problem, namely when the transport network
evolves dynamically.
IV. CONCLUSION
In this article, we proposed an online approach for the
stochastic shortest path problem, which computes a confidence
interval together with a shortest path. The approach is based
on an appropriate rescalling of the Iterate Averaging Stochastic
Gradient. Our main algorithm is a combination of the results
of [11] and of [13], leading to an efficient way to simulate
the distribution at time T of the shortest path in a stochastic
and potentially non-stationary (but smoothly evolving) en-
vironment. This algorithm has been implemented using the
Networkx library of the Python language, and obtained results
have been discussed.
In future work, the authors intention is to apply this online
2018 International Joint Conference on Neural Networks (IJCNN)