Network tomography has emerged as one of the lean approaches for efficient network monitoring, especially aiming at addressing the ever-increasing requirements for scaling and efficiency in modern network architectures and infrastructures. In this paper, we explore network coding and compressed sensing as enabling technologies in the context of network tomography. Both approaches capitalize on algebraic tools for achieving accuracy while allowing scaling of operation as the size of the monitored network increases. Initially, a brief overview of the tomographic problems and the related classification of methods is provided to better comprehend the problems encountered and solutions provided to date. Subsequently, we present representative approaches that employ either one of the aforementioned technologies and we comparatively describe their fundamental operation. Eventually, we provide a qualitative comparison of features and approaches that can be used for further research and technology development for network monitoring in future Internet infrastructures.

https://www.mdpi.com/1999-5903/12/2/20/pdf

A Review of Advanced Algebraic Approaches Enabling Network Tomography for Future Network Infrastructures

Continuous monitoring of network resources enables to make more-informed resource allocation decisions but incurs overheads. We investigate the trade-off between monitoring costs and benefits of accurate state information for a routing problem. In our approach link delays are modeled by Markov chains or hidden Markov models. The current delay information on a link can be obtained by actively monitoring this link at a fixed cost. At each time slot, the decision maker chooses to monitor a subset of links with the objective of minimizing a linear combination of long-run average delay and monitoring costs. This decision problem is modeled as a Markov decision process whose solution is computed numerically. In addition, in simple settings we prove that immediate monitoring cost and delay minimization leads to a threshold policy on a filter which sums up information from past measurements. The lightweight method as well as the optimal policy are tested on several use-cases. We demonstrate on an overlay of 30 nodes of RIPE Atlas that we obtain delay values close to the performance of the always best path with an extremely low monitoring effort when delays between nodes are modeled with hierarchical Dirichlet process hidden Markov models.

https://hal.archives-ouvertes.fr/hal-01857738/document

Joint Minimization of Monitoring Cost and Delay in Overlay Networks: Optimal Policies with a Markovian Approach

Recent research literature shows promising results by convolutional neural network- (CNN-) based approaches for estimation of traffic matrix of cloud networks using different architectures. Although conventionally, convolutional neural network-based approaches yield superior estimation; however, these rely on assumptions of availability of a large training dataset which is completely accurate and nonsparse. In real world, both these assumptions are problematic as training data size may be limited, and it is also prone to missing (or incomplete) measurements as well as may have measurement errors. Similarly, the 2-D training datasets derived from network topology based may be sparse. We investigate these challenges and develop a novel architecture which can cater for these challenges and deliver superior performance. Our approach shows promising results for traffic matrix estimation using convolutional neural network-based techniques in the presence of limited training data and outlier measurements.

/pdf/design-and-implementation-of-a-robust-convolutional-neural-4et45kjd9i.pdf

Design and Implementation of a Robust Convolutional Neural Network-Based Traffic Matrix Estimator for Cloud Networks

Many network inference (NI) problems are modeled as under-determined linear inverse problems where the number of observations or measurements are less than the number of unknowns. In this paper, a new technique for solving NI problems in dynamic network environments is presented. This technique is called optimal-coherent NI (OCNI) and it is applied in two stages. In the first stage, called learning phase, the optimal observation matrix (OOM) of network measurements are computed. In the second stage, called measurement and inference phase (MIP), the OOM is used to compute the least-norm solution and estimate the unknowns of interest. The OOM can be adaptively modified to improve the estimation accuracy. In this paper, first, the principles of OCNI are explained, and its properties are mathematically proved and experimentally justified. In addition, a new framework for traffic matrix estimation in software defined networks (SDN) is developed where the OCNI is the main technique for estimating the size of network flows. This framework is called OCcASION. Under the hard resource constraint of the size of ternary content addressable memory (TCAM) in SDN switches, OCcASION mainly uses the readily and reliably available link-load measurements to estimate the size of network flows where link-loads are provided via simple network management protocol (SNMP). In the learning phase, OCcASION computes the OOM of SNMP link-loads. In the MIP, OCcASION adaptively identifies and measures the most informative flows; moreover, it modifies the original OOM and accurately estimate the unknown traffic matrix. For this purpose, OCcASION uses the flexibility provided by the SDN to adaptively re-program a set of TCAM/flow-table entries of OpenFlow switches. The performance of OCcASION framework is evaluated using synthetic and real traffic traces of three practical networks topologies. It is shown that, this framework can significantly improve the accuracy of the traffic matrix estimation. For example, on Geant network the estimation error is approximately reduced by 83%, compared to regular minimum-norm estimation. Furthermore, the principles of OCNI is applied to estimate network link-delays where, in the learning phase, the OOM is computed using the network topology information. In the MIP, a set of path-delay measurements are measured at each measurement interval, and the OOM is used to coherently estimate unknown link-delays.

Optimal-Coherent Network Inference (OCNI): Principles and Applications

Network topology measurement is an important component in network research. Network tomography is able to accurately infer network topology by using end-to-end measurement without cooperation of internal routers. Unfortunately, traditional network tomography methods can not accurately estimate topology in the non-stationary network due to the variability of traffic distribution. In this paper, we present a novel network topology inference method based on subset structure fusion for accurate topology inference in the non-stationary network. First, we propose an end-to-end measurement method named three-packet to accurately probe the three-leaf-nodes subset structures of the network without the assumption that the packet delay or loss follows a stable distribution. Second, we propose a metric for the shared path length based on the structural characteristics of the subset structures to fuse these subset structures into a correct complete topology. The analytical and simulation results show that our method is more applicable for topology inference in the non-stationary network compared with the existing methods.

/pdf/network-topology-inference-based-on-subset-structure-fusion-47n016nnxb.pdf

Network Topology Inference Based on Subset Structure Fusion

Estimation of link delay densities in a computer network, from source-destination delay measurements, is of great importance in analyzing and improving the operation of the network. In this paper, we develop a general approach for estimating the density of the delay in any link of the network, based on continuous-time bivariate Markov chain modeling. The proposed approach also provides the estimates of the packet routing probability at each node, and the probability of each source-destination path in the network. In this approach, the states of one process of the bivariate Markov chain are associated with nodes of the network, while the other process serves as an underlying process that affects statistical properties of the node process. The node process is not Markov, and the sojourn time in each of its states is phase-type. Phase-type densities are dense in the set of densities with non-negative support. Hence, they can be used to approximate arbitrarily well any sojourn time distribution. Furthermore, the class of phase-type densities is closed under convolution and mixture operations. We adopt the expectation-maximization (EM) algorithm of Asmussen, Nerman, and Olsson for estimating the parameter of the bivariate Markov chain. We demonstrate the performance of the approach in a numerical study.

Neshat Etemadi Rad

Papers

Delay Network Tomography Using a Partially Observable Bivariate Markov Chain